-
Notifications
You must be signed in to change notification settings - Fork 2.1k
[Improve][Zeta] Add configuration option for physical DAG #9920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
…e/physical-dag
|
what's different of behavior of engine or job when enable or disable |
The engine or job behavior does not change depending on the |
|
I think we may also need to update this file: RestHttpGetCommandProcessor.java. |
So I think we should not add an config in seatunnel.yaml for this. We can only support it in restful api, WDYT? |
I agree. Should we support it in restful api and client too? |
LGTM :) |
…e/physical-dag
Purpose of this pull request
Introduce a new configuration option
physical-dag-enabledthat allows users to control whether the engine generates and executes a physical DAG (optimized for execution), or uses the logical DAG as defined in the configuration.Fix a bug in pipeline lookup logic where key comparison used different types, which could cause failures when mapping logical DAG vertices to pipelines.
Does this PR introduce any user-facing change?
Yes. Users can now use the new configuration option
physical-dag-enabledto control DAG execution mode.The default value maintains backward compatibility.
How was this patch tested?
I verified that
JobDAGInfooutputs differ betweenphysical-dag-enabled = trueandphysical-dag-enabled = falsewith a new test case.Check list
New License Guide