Patch pools should have an optional description #58066
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shubhamraj-git
requested review from
bbovenzi,
bugraoz93,
ephraimbuddy,
guan404ming,
jason810496,
pierrejeambrun,
rawwar and
ryanahamilton
as code owners
boring-cyborg
bot
added
area:API
shubhamraj-git
added
the
backport-to-v3-1-test
bugraoz93
approved these changes
Nov 8, 2025
jason810496
approved these changes
Nov 8, 2025
Member
jason810496
left a comment
jason810496
left a comment
There was a problem hiding this comment.
Thanks for the fix! LGTM on API side.
guan404ming
approved these changes
Nov 8, 2025
Member
guan404ming
left a comment
guan404ming
left a comment
There was a problem hiding this comment.
UI side looks nice and I think the CI failed is unrelated.
Contributor
Author
|
Updated branch to get the latest fix to solve the failing tests. |
guan404ming
force-pushed
the
fix-pools-page
branch
from
f8c94ec to
76dc0ff
Compare
shubhamraj-git
force-pushed
the
fix-pools-page
branch
from
76dc0ff to
2b31bda
Compare
github-actions bot
pushed a commit
that referenced
this pull request
Nov 10, 2025
(cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
Backport successfully created: v3-1-test
|
github-actions bot
pushed a commit
to potiuk/airflow
that referenced
this pull request
Nov 10, 2025
…8066) (cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
guan404ming
pushed a commit
that referenced
this pull request
Nov 12, 2025
…58169) (cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
ephraimbuddy
pushed a commit
that referenced
this pull request
Nov 18, 2025
…58169) (cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
ephraimbuddy
pushed a commit
that referenced
this pull request
Nov 19, 2025
…58169) (cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
ephraimbuddy
pushed a commit
that referenced
this pull request
Nov 19, 2025
…58169) (cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
ephraimbuddy
pushed a commit
that referenced
this pull request
Nov 20, 2025
…58169) (cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
hwang-cadent
added a commit
to hwang-cadent/airflow
that referenced
this pull request
Nov 22, 2025
…fully test (#1) * Enable PT006 rule to 15 files in helm-tests (other,security) (#57844) Co-authored-by: Kalyan R <[email protected]> * Airflow 3.1.2 has been released (#57887) * Enable PT006 rule to airflow-core tests (utils) (#57885) Co-authored-by: Kalyan R <[email protected]> * Use Iterable instead of list in API responses (#57878) * Enable PT006 rule to task-sdk tests (#57841) Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to airflow-core tests (security, lineage, jobs, executors, datasets) (#57913) Co-authored-by: Kalyan R <[email protected]> * Make set_xcom route in API server DRY by reusing logic of XComModel.set method (#55289) Co-authored-by: Tzu-ping Chung <[email protected]> * Enable PT006 rule to 17 files in providers (operatorsproviders/amazon/tests/unit/amazon/aws/operators/) (#57903) Co-authored-by: Kalyan R <[email protected]> * Add number of queries guard for ui config (#57868) * Enable PT006 rule to airflow-core tests (ti_deps, serialization) (#57911) Co-authored-by: Kalyan R <[email protected]> * Unify collection.abc imports (#57914) * Expanding task sdk integration tests to cover critical xcom operations (#57797) * Enable PT006 rule to airflow-core tests(triggers, timetables) (#57888) Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * Enable PT006 rule to airflow-core tests (utils) (#57886) Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * Add required context messages to all DagRun state change notifications (#56272) * Propogate dagrun timeout failure context * add unit test * pass started msg for notify dagrun start state change * make msg required for notify_dagrun_state_changed * fix type unit test failure * Fixing callback tests due to wrong types (#57935) * Bump prek, zizmor, go to fix the CI (#57930) * Enable PT006 rule to openai Provider test (#57920) * Fix MyPy errors in airflow-core/tests/unit/api_fastapi/core_api/routes/public (#57230) * Fix scheduler using stale max_active_runs from SerializedDAG (#57619) * Fix scheduler using stale max_active_runs from SerializedDAG When max_active_runs is updated via versioned bundles, the scheduler was checking the limit against stale SerializedDAG data instead of the latest DagModel value. This caused queued DAG runs to be incorrectly blocked even when the limit had been increased. The scheduler now uses max_active_runs from DagModel (accessed via dag_run association proxy) to ensure versioned bundle updates to max_active_runs are respected immediately. closes: #57604 * fixup! Fix scheduler using stale max_active_runs from SerializedDAG * fixup! fixup! Fix scheduler using stale max_active_runs from SerializedDAG * feat: use get async conn from common compact (#57894) * Expanding task sdk integration tests to test connection operations (#57805) * Enable PT006 rule to keycloak Provider test (#57923) * Expanding task sdk integration tests to test variable get operation (#57802) * Add secret masking for Jinja template rendering exceptions (#57467) * feat(openlineage): Add parentRunFacet for DAG events (#57809) * Decrease the batch inference size for example_bedrock_batch_inference (#57912) The system tests just needs to run the minimum batch size to test functionality. The shorter batch should decrease test runtime and resource usage. Also fix a bug that caused the prompts list to be an unreliable size, there was a missing flush on the temp file. * Add number of queries guard for ui dependencies (#57957) * Add number of queries guard for ui dashboard (#57956) * Migrate FAB DELETE /roles to FastAPI (#57780) * fix: MyPy type errors in pool.py (#57810) * Enable PT006 rule to google Provider test （hooks） (#57915) * Enable PT006 rule to google Provider test * ads * /cloud/hooks * fix ci test * Update AWS auth manager documentation to fix login callback URL (#57974) * Enable PT006 rule to jdbc Provider test (#57919) * Enable PT006 rule to http Provider test (#57917) * Enable PT006 rule to http Provider test * fix ci error * Enable PT006 rule to jenkins Provider test # (#57922) * Enable PT006 rule to hashicorp Provider test (#57916) * Add `LIST` permission to admin role in Keycloak auth manager (#57978) * Enable PT006 rule to google Provider test (operators part1) (#57943) * modify test_variables Co-authored-by: Kalyan R <[email protected]> Signed-off-by: rich7420 <[email protected]> * fix ci test --------- Signed-off-by: rich7420 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * modify test_variables (#57945) Signed-off-by: rich7420 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to trino Provider test (#57931) * Remove global from Fernet Crypto tooling (#57642) * Remove global from Fernet Crypto tooling * Add support for key rotation * Fix pytests * Fix pytests * Fix pytests * Update the version of postgres we test against (#57899) v13 is reaching EOL in a few weeks (well before we will release 3.2, certainly) and v18 is out now, so we should test against it * Add `breeze ci upgrade` command to automate CI infrastructure upgrade (#57958) We have some semi-complex set of tools that take care about upgrading of all our CI infrastructure periodically. While dependabot has a lot of use, a lot of cases is not handled yet in it - such as upgrading the charts, important dependencies in our scripts, dockerfiles and so on. While we already have a set of prek hooks and github Actions that do a lot there, some of that has to be done periodically outside of dependabot - this CI wraps a number of those upgrade tools into a single `ci upgrade` command that can be run locally, generate and open PR - and can be used in both - main and any of the v*test branches. Future improvement areas: * docker container upgrades for Helm chart and docker compose * slack notification after such upgrade PR is opened * more ..... * Enable ruff PLW1509 rule (#57659) * Enable ruff PLW1509 rule * Fix pytests * Update ORM for asset / dag partitioning (#57360) We add string field partition_key to models DagRun and AssetEvent so these entities can be partition aware. And we add new models AssetPartitionDagRun and PartitionedAssetKeyLog. Think of AssetPartitionDagRun as a provisional dag run. This record is created when there's an asset event that contributes to the creation of a dag run for this dag_id / partition_key combo. It may need to wait for other events before it's ready to be created though, and the scheduler will make this determination. The PartitionedAssetKeyLog entity is a mapping between AssetPartitionDagRun and its corresponding AssetEvent entities. * KubernetesPodTriggerer reads pod logs instead of KubernetesPodOperator (#57531) * Move container-related functions from PodManager to a separate file * Moved unit tests * Sync and async workflow use the same code to track Pod startup * Reworked unit tests and pod startup logic * Add api permission error detection for triggerer * Fix pytest fixture * Removed not requried code * Move log file reading to triggerer * Removed not required code * Removed not requried unit test * Fixed unit tests * Clean up unit tests * Adapt unit test * Adapt return type --------- Co-authored-by: AutomationDev85 <AutomationDev85> * Further fixes for release preparation process for tarballs (#57996) When preparing tarballs for airflow-ctl, airflow, even if we are preparing tarballs for "final" version we need to prepare them from a tag that has version suffix added. And we need to know the suffix. The prepare-tarball command has now --version-suffix added as well as the VERSION, VERSION_SUFFIX and VERSION_RC variables have been synchronized to be the same for airflow, airflowctl, helm-chart - previously we used VERSION to contain also RC and VERSION_WITHOUT_RC to be final version - this is now synchronized across all release instructions. * Remove unnecessary `--tag` flag from git push (#57999) * Use VERSION_SUFFIX in airflow-ctl when preparing PyPI packages. (#58001) * build: upgrade ruff to 0.14.4 (#58017) * Expanding task sdk integration tests to test few dagrun operations (#57955) * Adjustments in release docs (#58008) * Fix Connection test failures due to Fernet key caching (#58034) * fix: Add .txt extension for log download function (#57991) * Fix double redirection while authenticating in Fab auth manager (#57993) * Migrate FAB GET /roles/{name} to FastAPI (#58009) * Fix mypy type errors in test_taskinstance.py (#57942) * fix mypy type errors in test_taskinstance.py * Use tuple * Fix logout in airflow-core (#57990) * Enable PT006 rule to microsoft Provider test(hooks) (#57932) * Enable PT006 rule to ssh Provider test (#57929) * Enable PT006 rule to microsoft Provider test(log,sensors) (#57927) * Enable PT006 rule to samba Provider test(transfers, hooks) (#57926) * Enable PT006 rule to samba Provider test * fix static check * Enable PT006 rule to microsoft Provider test(transfers) (#57925) * PT006 modify standard (operator) (#58020) * Enable PT006 rule to standard Provider test(decorator, hook) 8 files (#58019) * Enable PT006 rule to 6 files in providers (edge3,git) (#58018) * modify all remaining files related to airflow-core (#58015) * Enable PT006 rule to airflow-core tests (auth, common, always, integration) (#58014) * modify route/public (#58012) * Enable PT006 rule to airflow-core tests (route/ui, execution_api) (#58010) * Enable PT006 rule to airflow-core tests (cli) (#58007) * Enable PT006 rule to 23 files in providers (all remaining files related to amazon) (#58005) * Enable PT006 rule to 23 files in providers (amazon -> hooks, links, log, queues) (#58003) * Fix logout in Fab and Keycloak auth managers (#57992) * Stubbing missing tests for task sdk integration testing for xcoms (#58031) * Stubbing missing tests for task sdk integration testing for variables (#58030) * Stubbing missing tests for task sdk integration testing for connection (#58029) * Stubbing missing tests for task sdk integration testing for dagruns (#58036) * Docs: Add note on running prek inside Breeze (#58039) * Docs: Add note on running prek inside Breeze * Update dev/breeze/doc/03_developer_tasks.rst --------- Co-authored-by: Jarek Potiuk <[email protected]> * Update Arabic translation 2025-11-07 (#58042) * Add number of queries guard for ui team (#57980) * Add number of queries guard for ui grid (#57977) * fix typos in comments within triggerer_job_runner.py (#58013) * Update Hebrew translation 2025-11-07 (#58041) * Add number of queries guard for ui calendar (#58044) * Enable PT006 rule to 14 files in providers (databricks,dbt,docker) (#57994) Signed-off-by: Anusha Kovi <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to sftp Provider test (#57924) * Enable PT006 rule to microsoft Provider test(operators) (#57928) * Enable PT006 rule to postgres Provider test (#57934) * Enable PT006 rule to microsoft Provider test(azure,mssql,psrp) (#57936) * Enable PT006 rule to mysql Provider test (#57937) * Enable PT006 rule to celery Provider test (#57938) * Enable PT006 rule to neo4j Provider test (#57939) * Enable PT006 rule to openlineage Provider test (#57940) * Enable PT006 rule to oracle Provider test (#57941) * Enable PT006 rule to google Provider test (operators part2) (#57944) * modify test_variables Co-authored-by: Kalyan R <[email protected]> Signed-off-by: rich7420 <[email protected]> * fix for the ci test --------- Signed-off-by: rich7420 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to google Provider test (log,openlineage,common,utils) (#57947) Signed-off-by: rich7420 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to airflow-core tests (core, dag_processing) (#57948) Signed-off-by: Xch1 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to airflow-core tests (models) (#57949) * modify test_asset Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_backfill Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_callback Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_cleartasks Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_connection Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_dag Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_dagrun Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_deadline Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> --------- Signed-off-by: Xch1 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to google Provider test (triggers) (#57950) Signed-off-by: rich7420 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to airflow-core tests (models) (#57951) Signed-off-by: Xch1 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to google Provider test (triggers) (#57953) Signed-off-by: rich7420 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to slack Provider test (#57963) Signed-off-by: Your Name <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to 19 files in providers (airbyte, alibaba, atlassian, papermill, presto, redis, singularity, sqlite, tableau, vertica, weaviate, elasticsearch, exasol) (#57986) Signed-off-by: Anusha Kovi <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to 19 files in providers (cncf,common) (#57995) Signed-off-by: Anusha Kovi <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to 13 files in providers (apache) (#57998) Signed-off-by: Anusha Kovi <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Enable PT006 rule to standard Provider test(ssensor, trigge, util) 9 files (#58022) Signed-off-by: Your Name <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Add number of queries guard for ui next_run_assets (#58052) * fix MyPy type errors in datamodels/hitl.py (#57808) * Remove global from serialization (#57703) * Remove global from serialization * Fix connection test and fernet instability * Enable PT006 rule to airflow-core tests (route/public) (#58011) Signed-off-by: Xch1 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Fix MyPy type errors in api_fastapi(security.py),airflow/models(dag_version.py,hitl.py),mark_tasks.py (#58054) * Fix duplicated SQLAlchemy sessions caused transactions fail to close (#57815) * feat: add resize function to dag run / TI notes (#57897) * feat: add resize funtion to notes * refactor: deduplicate markdown dialog storage key constant * Fix broken canary, remove unneeded crypto key generation (#58074) * Remove global from kerberos (#57702) * Remove global from kerberos * Review feedback to adjust pytest using a fixture * Apply suggestion from @potiuk Co-authored-by: Jarek Potiuk <[email protected]> --------- Co-authored-by: Jarek Potiuk <[email protected]> * Add test setup and example asset operation tests to task SDK integration tests (#58027) * Add number of queries guard for ui structure (#58051) * Expanding task sdk integration tests to test asset operations (#58028) * update CODEOWNERS (#58075) * Fix reproducibility check source tarball instructions (#58077) * Remove global from settings part1 (#57705) * add doc line for hot reloading ui in certain wsl environments (#57921) * docs: add doc for hot reloading in certain wsl environments * typo * Update contributing-docs/15_node_environment_setup.rst --------- Co-authored-by: Elad Kalif <[email protected]> * Add learnings from Airflow 3 migration (#57989) * Remove unused NullFernet from Crypto (#57988) * Fix release cleanup for providers (#58082) * Fix release cleanup for providers * Apply suggestion from @potiuk * Fix ts-compile-lint-ui hook not finding any UI files (#58099) * Update providers metadata 2025-11-08 (#58084) * The e2e-tests workflows properly use workflow-name to pass name (#58100) The workflows used `inputs.name` but the passed input was really the `workflow-name` one. * Revert "Fix duplicated SQLAlchemy sessions caused transactions fail to close (#57815)" (#58097) This reverts commit e5cb168fafe40059233ed9ab5fa191f1958a89fd. * Revert "Remove unused NullFernet from Crypto (#57988)" (#58107) This reverts commit c47c0817d8e11f885f4eef91232c08e1469e8c95. * Upgrade hungarian language package (#58104) * Upgrade hungarian language package * fix linting * fix linting * Add `executor.running_dags` gauge to expose count of running DAGs (#12368) (#52815) * feature: introducing a counter for number of DAGs in running state * wrap emit executor.running_dags gague as a function * add test for _emit_running_dags_metric * add executor.running_dags to metrics.rst * use Sqlalchemy 2.0 style * check for metrics configuration before emitting running dags metric * Enable metrics configuration for running dags in DagFileProcessorManager test * Add configuration for multi-team support in test_bundles_with_team * add go-sdk/bin/.gitignore back * Update .gitignore to include all files except itself * Refactor running DAGs metric emission: move to SchedulerJobRunner and update tests * Add running DAGs metric and remove unused metrics check * Add dagrun_metrics_interval to config and update running DAGs metric to use 'scheduler' namespace * Remove unused import of DagRunState in test_emit_running_dags_metric * Fix Low dep tests:core * Rename running DAGs metric to 'scheduler.dagruns.running' for consistency and update related test assertions * Components of providers docs update (#57657) * components of providers docs update * comes installed * static check reproducible * added hooks, triggers and etc cmt * rebase regen files * reproducible static check * AIP-67 - Multi-team: Per team executor config (#57910) This adds support for reading team specific config sections from the config file. This allows multiple teams to configure separate instances of the same type of executor. * Enable ruff PLW2101,PLW2901,PLW3301 rule (#57700) * Enable ruff PLW2901 rule * Fix follow-up mypy complaints after PLW2901 rule * Fix tests for pools and infinite size * Fix logic in bigquery * Fix variable renaming * Fix mypy alert * Review feedback by Wei * Fix mypy * Revert accidentially deleted line * Second pass review suggestions by Wei * Add registry.secretNames and registry.connections options to Helm chart (#58094) Co-authored-by: Your Name <[email protected]> * Fix Python 3.13 RuntimeWarning in test_reading_from_pipes (#58132) * fix: Rendered Templates not showing dictionary items in AF3 (#58071) * Rendered Templates not showing dictionary items in AF3 * add a unit test * fix test error * Airflow documentation updates for Windows WSL2 installation (#57027) * Update start.rst for Windows WSL2 installation Airflow Installation documentation updates for Windows using WSL2 * Update start.rst Fixed typo in WSL * Update start.rst with formatting changes Formatting changes for Airflows Windows WSL2 installation * Update start.rst to add line after ..code-block:: bash Update start.rst to add line after ..code-block:: bash * Fix trailing whitespace in start.rst * Update airflow-core/docs/start.rst Co-authored-by: Jarek Potiuk <[email protected]> * Update airflow-core/docs/start.rst Co-authored-by: Jarek Potiuk <[email protected]> * Update airflow-core/docs/start.rst Co-authored-by: Jarek Potiuk <[email protected]> * Fix trailing whitespace, reshuffling content, adding uv in start.rst * Incorporating review comments for performance benefits of uv over pip in start.rst * Fix RST syntax errors: add :: to code-block and fix path separators * Fix typos in link --------- Co-authored-by: Jarek Potiuk <[email protected]> * fix: HITL params not validating (#57547) * RFC: Close portuguese gap airflow 3.1 (#58119) * close portuguese gap airflow 3.1 * fix lint * close catalan gap (#58110) Co-authored-by: Bugra Ozturk <[email protected]> * Add CLI hot-reload support via --dev flag (#57741) * Add watchfiles dependency and hot-reload utility with --dev flag support Co-authored-by: jason810496 <[email protected]> * Add tests for --dev flag and hot-reload functionality Co-authored-by: jason810496 <[email protected]> * Fix watchfiles API usage with proper DefaultFilter Co-authored-by: jason810496 <[email protected]> * Address code review feedback: fix help text, API usage, and improve error messages Co-authored-by: jason810496 <[email protected]> * Final code review fixes: improve logging and test clarity Co-authored-by: jason810496 <[email protected]> * Refactor hot_reload utils * Refactor hot-reload: move to cli module, remove pyproject change, add dag-processor support Co-authored-by: jason810496 <[email protected]> * Fix serve-log teardown issue using psutil to terminate process tree Co-authored-by: jason810496 <[email protected]> * Refactor hot_reload: extract _terminate_process_tree helper function Co-authored-by: jason810496 <[email protected]> * Add type annotations and process_name parameter to hot_reload Co-authored-by: jason810496 <[email protected]> * Refactor final nits - remove logger in triggerer command - ensure type annotation in hot-reload module - fix test hot-reload * Fix mypy error * Respect DEV_MODE env var * Fix nits --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> * Fix Connection test failures due to Fernet key caching (#58137) * Convert all airflow distributions to be compliant with ASF requirements (#58138) According to https://www.apache.org/legal/src-headers.html#faq-binaries all our released source packages and binaries should have both LICENCE and NOTICE files - even if we do not have any 3rd-party software embedded. This PR updates it accross the board and makes sure that all our distributions contain both LICENCE and NOTICE file and that they are added as "licence" information to all our distribution packages when they are built. This PR also modernizes all our pyproject.toml files with regards to latest metadata for licenses: * license-file is an array of glob patterns * license is s license expression * the deprecated Trove Classifier is removed Inspired by #58131 * fix mypy errors in delete_dag.py (#58101) * close spanish gap airflow 3.1 (#58117) * close spanish gap airflow 3.1 * fix lint * Update Release instruction to include Task SDK version update (#58134) * Update Release instruction to include Task SDK version update This PR updates the release branch to include task-sdk version update when doing a new release. Also updates how we sync test branch for a new release * fixup! Update Release instruction to include Task SDK version update * Update dev/README_RELEASE_AIRFLOW.md * Updates the go-sdk/Justfile for the changed package layout (#58153) * Remove deprecation warning in common test utils (#58152) * Remove pytest deprecation warning in common test utils > pytest.PytestRemovedIn9Warning: Marks applied to fixtures have no effect * Remove mark-on-fixture in AWS test too Every use of that fixture already had this mark, so we remove it * fix mypy errors in scheduler_job_runner.py (#58167) * CI: Upgrade important CI environment (#58164) * Patch pools should have an optional description (#58066) * Update UI to allow creation of DagRuns with partition key (#58004) Allow triggering dag runs for a specific partition key. When there's a partition key associated with a dag run, show it in the UI. * Bump boto3 from 1.40.68 to 1.40.69 in /dev/breeze (#58160) Bumps [boto3](https://github.com/boto/boto3) from 1.40.68 to 1.40.69. - [Release notes](https://github.com/boto/boto3/releases) - [Commits](https://github.com/boto/boto3/compare/1.40.68...1.40.69) --- updated-dependencies: - dependency-name: boto3 dependency-version: 1.40.69 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Remove unnecessary list (#58141) * RFC: Close italian gap airflow 3.1 (#58120) * close italian gap airflow 3.1 * fix lint * Indent todo comment for IDE highlighting compatibility (#58174) * Fix CI upgrade script to not delete branch from origin remote (#58163) * Upgrade email notifications to use SmtpNotifier (#57354) * Group upgrades of python dependencies (#58162) Rather than having multiple PRs - separately for each dependency upgrade, we should group the dependency upgrades accross several distributions into single PRs as much as possible. * Fix Enum-str interpolation in callback metrics in Python 3.11+ (#58168) Followup to https://github.com/apache/airflow/pull/57215#pullrequestreview-3426156849 Even though CallbackState is a string Enum and works fine with string comparisons, the default __str__ method was changed in Python 3.11 causing test failures due to unexpected metric name: airflow-core/tests/unit/models/test_callback.py:105: in test_get_metric_info assert metric_info["stat"] == "deadline_alerts.callback_success" E AssertionError: assert equals failed E 'deadline_alerts.callback_CallbackState.SUCCESS' 'deadline_alerts.callback_success' This overrides __str__ in CallbackState so it behaves consistently across python versions. * fix(migration): disable sqlite fkeys (#58130) * fix(migration): ignore sqlite fk constraint * fixup! fix(migration): ignore sqlite fk constraint * fixup! fixup! fix(migration): ignore sqlite fk constraint * fixup! fixup! fixup! fix(migration): ignore sqlite fk constraint * Fix location of comment in dags_needing_dagruns (#58173) The comment applies only when we hit AttributeError, so moving it there makes sense. (cherry picked from commit 2a2c1b1c8517bed956e9b93c34ee3338b2449a5c) * Create additional users in breeze for testing (#58126) * feat: added --create-all-roles flag to breeze. creates test users with each role by default for SimpleAuthManager * update command config * ci: ran prek --all-files * remove dev modecreate roles error * remove --dev-mode requirement for breeze start-airflow --create-all-roles * docs: remove documentation * Add HTTP URL authentication support to GitHook (#58194) GitHook now supports authentication tokens with HTTP URLs, matching the existing HTTPS support. This enables authentication with internal git repositories that use HTTP instead of HTTPS. HTTP URLs without authentication tokens are preserved as-is. * Add push_logs configuration option to Edge executor (#58125) * Add push_logs configuration option to Edge executor Adds a new boolean configuration option AIRFLOW__EDGE__PUSH_LOGS to control whether edge workers push log files to the central site. The option is set to True by default to maintain existing behavior. When disabled, edge workers will keep logs locally without uploading them to the central Airflow site, which can be useful for: - Reducing network bandwidth usage - Keeping logs on edge sites for compliance/security reasons - Troubleshooting network connectivity issues * Add version_added: 1.5.0 to push_logs configuration option * Update release instructions for Airflow Ctl (#58206) * Rename task sdk integration tests folder hierarchy correctly (#58193) * Fix the `__init__.py` for tests (#58211) The `__init__.py` for tests should have the legacy namespaces defined in order for IDE imports to work properly. This works also in IDEs where `tests` folders are added the path of ours (this is done via `setup_idea.py` and `setup_vscode.py`). There was a bug in our prek hook wher it did not have "expected" and "namespace" __init__.py for those known test folders. This PR fixes the prek hook and some of the recently added folders that had wrong __init__.py added. * Make sure regenerating provider dependencies happens only once (#58198) This has to be done in global_constants module, because regeneration can happen just during importing and if we try to do it in a separate package circular dependencies might happen. * Fix check_files.py script after source tar was renamed (#58216) * Re-Enable Back-Compat tests for Edge3 (#58217) * Add flatten_structure parameter to GCSToS3Operator (#56134) (#57713) * Add flatten_structure parameter to GCSToS3Operator * Enhance GCSToS3Operator docs clarity and refactor tests to use pytest.mark.parametrize for better maintainability * Improve GCSToS3Operator documentation consistency and add concrete examples - Use consistent "takes precedence over" terminology in warning messages - Add specific file path transformation example in class docstring * Fix GCSToS3Operator test log mocking for Airflow SDK compatibility - Replace mock.patch.object() with mock.patch() string path to handle - read-only log property in new Airflow SDK. * Add stream method to RemoteIO (#54813) * Add stream method to RemoteIO * Import BaseOperator from version_compact * Skip Airflow 2 compact test for RemoteIO * Add a new Protocal for .stream method, check attr in runtime * Remove NotImplementedError * Clearify naming for log response types * Fix comment for LogMessages * Try to make mypy happy * Remove redundant tests * Check hasattr before getattr * Fix nits for using getattr * i18n(Ko): add missing translations(Nov 12) (#58222) * Improve is_container annotation (#58224) Using TypeGuard lets the function perform type-narrowing for us, so usages after it would not need a custom cast. The downside is we now need to specify a bound, but we don't actually use this function that often anyway and can just list all the possibilities. * Fix MSSQLToGCSOperator MSSQL BIT data type conversion to Parquet boolean (#57514) * Fix MSSQLToGCSOperator MSSQL BIT data type conversion to Parquet boolean Changes type_map from "BOOLEAN" to "BOOL" to match the expected schema type in BaseSQLToGCSOperator._convert_parquet_schema(). This fixes issue #57461 where exporting MSSQL bit fields to Parquet format would raise ArrowTypeError: Expected bytes, got a 'bool' object. * Add a test case to verify that MSSQL BIT fields are mapped to boolean in the parquet schema * Fix duplicated SQLAlchemy sessions caused transactions fail to close (#58196) * Make `conf.validate` pluggable on config parser (#58188) * Rewrite config parser get() with modular lookup sequence (#57970) * Update pyproject.toml files with pytest>=9.0.0 TOML syntax (#58182) * Enable PT006 rule to task-sdk tests (#57842) * modify test_asset_decorators Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_asset Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify _internal/test_templater Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify _internal/test_decorators Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_sensor Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_operator Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * modify test_client Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> --------- Signed-off-by: Xch1 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Skip the team config test until new modular config supports it (#58233) * Fix a few typos in release process for Airflow (#58219) * Enable pt006 rule and fix new generate errors (#58238) * modify remaining new generated PT006 errors Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> * delete PT006 ignore list Co-authored-by: Kalyan R <[email protected]> Signed-off-by: Xch1 <[email protected]> --------- Signed-off-by: Xch1 <[email protected]> Co-authored-by: Kalyan R <[email protected]> * Delete all unnecessary LICENSE Files (#58191) When rebasing/working on #58138 I **thought** I deleted those deeply nested LICENSE files in our providers that were installed together with other files when packages were installed. But since the files are now embedded as metadata in the packages and this is the only requirement from the ASF point of view, we can safely delete them now. This is continuation of #58138 * Fix generated Go header in code (#58243) * CI: Upgrade important CI environment (#58245) * Add protoc pinning and include it in important versions upgrade (#58244) * fix mypy errors in airflow-core/src/airflow/jobs/job.py (#58190) * Add check for schedule parameter for system tests (#58254) * Support for "reconnecting" Supervisor Comms from task process when `dag.test()` is used (#58147) This is a follow up to #57212, which worked fine "at run time" but did not work in many of our own unit tests, which rely on `dag.test` or `ti.run`. The way this is implemented is that when we use the InProcessTestSupervisor we pre-emptively create a socket pair. We have to create it even it its not being used, as we can't know. And since this is all in one process we create a thread to handle the socket comms. Since this is only ever for tests performance or hitting the GIL doesn't matter. * providers/trino: set split_statements=True by default and add unit tests (#58158) * providers/trino: set split_statements=True by default in TrinoHook.run() * providers/trino: add unit tests for TrinoHook.run() * providers/trino: fixed patches in split_statement tests * providers/trino fix documentation * providers/trino: Apply ruff auto-format fixes in TrinoHook * providers/trino: Apply ruff-format fixes in TrinoHook * providers/trino: re-run after rebase on upstream/main --------- Co-authored-by: Nikita <[email protected]> Co-authored-by: Nikita.Kalganov <[email protected]> * Fix `KubernetesPodOperator` fails to delete pods with None value labels (#53477) * Add a `schedule` parameter to the Asana example dag (#58267) * fix: add required arguments when creating an external table (#58199) * Fix main failing on cross-merged PR (#58270) * Exclude sagemaker for Python 3.13 due to pydanamodb pinning old sqlean (#58262) * Fix test_async_write_logs_should_execute_successfully test - Add client mock patch to properly mock self.client.read_namespaced_pod_log - Fix assertions to verify actual log output using caplog - Fix get_logs=False assertion to check client.read_namespaced_pod_log instead of pod_manager.read_pod_logs - Add await_pod_completion mock return value Fixes #57515 * issue-58236: Adding gcp_conn_id to template_fields for BigQuery and Dataproc Operators (#58250) * Make it easier and faster to iterate on task-sdk-integration-tests (#58231) There were a few of things that made task-sdk-tests iteration a bit unobvious and slower. With this PR, we should be able to iterate over task-sdk-integration-tests WAY faster and get more contributors involved in contributing to those. * It was not clear that prerequisite of running the tests was building PROD image for Pyton 3.10. This is now clear in the documentation. * PROD images can be built in two different modes - from sources with --installation-method equal to . or from packages with the --installatio-method equal to "apache-airflow". This was not clearly communicated during build and it is now printed at output * It was not clear that when you build PROD images from sources, you should first compile ui assets, because otehrwise the assets are not added as part of the image. With this PR the `breeze prod-image build` command checks if the .vite manifest is present in the right `dist` folders and will error out, suggesting to run `breeze compile-ui-assets` before. * If the PROD image has not been built before, breeze will propose to build it and even do it automatically if the answer is not provided within 20 seconds. * when building PROD images from sources, it is faster to rebuild the images with `uv` than with `pip`. the --use-uv parameter now defaults to False when building from packages and to True when building from sources. * There was an error in .dockerignore where generated dist files were not added to context when PROD image was built from sources. This resulted in "permission denied' when such PROD images were used to run tests. * The test compose had fallback of Airflow 3.0.3 which would be misleading if it happened. Now, AIRFLOW_IMAGE_NAME is mandatory * We are now mounting sources of Airflow to inside the image by default and skip it in CI. This mounting happens in local environment where PROD image is built usually from sources, and it is disabled in CI by using --skip-mounting-local-volumes flag. We also do not stop docker compose by default when runnig it locally in order to make fast iteration the default. * We pass host operating system when starting the compose, and we only change ownership on Linux - this is a long running operation on MacOS because mounted filesystem is slow, but it's also not needed on MacOS because the file system also maps ownershipt and files created by Airflow are created with local user id. * We pass local user id to containers to make sure that the files created on linux are created by the local user (logs and the like). * We are now detecting whether docker-compose is running and when we run with locally mounted sources, we reuse those running containers. When we don't mount local sources, we shut-down the compose before running to make sure we do not have sources mounted - and we close the compose by default when we do not mount local sources. * When sources are mounted we are only enabling DEV_MODE inside the containers so that components are hot-reloading (new feature added in #57741 last weeks. This way you do not have to restart anything when sources are changed and you can re-run the tests when docker compose is running. * The environment are passsed now via .env file so that you can easily reproduce docke compose command locally * The docker compose files are not copied any more, they are moved directly to the top of 'task-sdk-integraiton-tests' and used from there. * A `--down` flag is added to breeze testing task-sdk-integration-tests that tears down running docker compose. * Additional diagnostics added to show what's going on. * Handling verbose option from breeze by adding more debugging information * Updated documentation about the tests to be more comprehensive about the options, updated file structure etc. * Small QOL immprovement - if expected dags are not yet parsed by dag file processor, when test starts, getting their status will return 404 "Not Found". In such case our tests implemented a short retry scheme with tenacity * Add working_directory parameter (#58210) * Add working_directory parameter * fix test * restore import Connection * ignore type check * Fix atomicity issue in SerializedDagModel.write_dag preventing orphaned DagVersions (#58259) * KubernetesExecutor: retry pod creation on Kubernetes API 500 errors (#57054) Treat 500 InternalServerError responses from the Kubernetes API as transient when creating worker pods and requeue the task according to `task_publish_max_retries` (or indefinitely when -1). This avoids failing tasks due to transient kube-apiserver errors and logs retry attempts with details. * Fix walking through wildcarded directory in FileTrigger (#57155) Fix is similar to the one in #21729 * Identify duplicate kubernetes section airflow configuration and mark them as deprecated (#57028) * Bump the edge-ui-package-updates group across 1 directory with 19 updates (#58235) * Bump the edge-ui-package-updates group across 1 directory with 19 updates Bumps the edge-ui-package-updates group with 19 updates in the /providers/edge3/src/airflow/providers/edge3/plugins/www directory: | Package | From | To | | --- | --- | --- | | [@chakra-ui/react](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/react) | `3.28.0` | `3.29.0` | | [@tanstack/react-query](https://github.com/TanStack/query/tree/HEAD/packages/react-query) | `5.90.5` | `5.90.8` | | [axios](https://github.com/axios/axios) | `1.12.2` | `1.13.2` | | [react-router-dom](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router-dom) | `7.9.4` | `7.9.5` | | [@eslint/compat](https://github.com/eslint/rewrite/tree/HEAD/packages/compat) | `1.4.0` | `1.4.1` | | [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js) | `9.38.0` | `9.39.1` | | [@trivago/prettier-plugin-sort-imports](https://github.com/trivago/prettier-plugin-sort-imports) | `5.2.2` | `6.0.0` | | [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `24.9.1` | `24.10.1` | | [@types/react](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react) | `19.2.2` | `19.2.4` | | [@types/react-dom](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react-dom) | `19.2.2` | `19.2.3` | | [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/tree/HEAD/packages/plugin-react-swc) | `4.2.0` | `4.2.2` | | [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) | `4.0.3` | `4.0.8` | | [eslint](https://github.com/eslint/eslint) | `9.38.0` | `9.39.1` | | [globals](https://github.com/sindresorhus/globals) | `16.4.0` | `16.5.0` | | [happy-dom](https://github.com/capricorn86/happy-dom) | `20.0.8` | `20.0.10` | | [typescript](https://github.com/microsoft/TypeScript) | `5.8.3` | `5.9.3` | | [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) | `8.46.2` | `8.46.4` | | [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `7.1.12` | `7.2.2` | | [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) | `4.0.3` | `4.0.8` | Updates `@chakra-ui/react` from 3.28.0 to 3.29.0 - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/react/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/[email protected]/packages/react) Updates `@tanstack/react-query` from 5.90.5 to 5.90.8 - [Release notes](https://github.com/TanStack/query/releases) - [Changelog](https://github.com/TanStack/query/blob/main/packages/react-query/CHANGELOG.md) - [Commits](https://github.com/TanStack/query/commits/@tanstack/[email protected]/packages/react-query) Updates `axios` from 1.12.2 to 1.13.2 - [Release notes](https://github.com/axios/axios/releases) - [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md) - [Commits](https://github.com/axios/axios/compare/v1.12.2...v1.13.2) Updates `react-router-dom` from 7.9.4 to 7.9.5 - [Release notes](https://github.com/remix-run/react-router/releases) - [Changelog](https://github.com/remix-run/react-router/blob/main/packages/react-router-dom/CHANGELOG.md) - [Commits](https://github.com/remix-run/react-router/commits/[email protected]/packages/react-router-dom) Updates `@eslint/compat` from 1.4.0 to 1.4.1 - [Release notes](https://github.com/eslint/rewrite/releases) - [Changelog](https://github.com/eslint/rewrite/blob/main/packages/compat/CHANGELOG.md) - [Commits](https://github.com/eslint/rewrite/commits/compat-v1.4.1/packages/compat) Updates `@eslint/js` from 9.38.0 to 9.39.1 - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](https://github.com/eslint/eslint/commits/v9.39.1/packages/js) Updates `@trivago/prettier-plugin-sort-imports` from 5.2.2 to 6.0.0 - [Release notes](https://github.com/trivago/prettier-plugin-sort-imports/releases) - [Changelog](https://github.com/trivago/prettier-plugin-sort-imports/blob/main/CHANGELOG.md) - [Commits](https://github.com/trivago/prettier-plugin-sort-imports/compare/v5.2.2...v6.0.0) Updates `@types/node` from 24.9.1 to 24.10.1 - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) Updates `@types/react` from 19.2.2 to 19.2.4 - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react) Updates `@types/react-dom` from 19.2.2 to 19.2.3 - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react-dom) Updates `@vitejs/plugin-react-swc` from 4.2.0 to 4.2.2 - [Release notes](https://github.com/vitejs/vite-plugin-react/releases) - [Changelog](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite-plugin-react/commits/[email protected]/packages/plugin-react-swc) Updates `@vitest/coverage-v8` from 4.0.3 to 4.0.8 - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.0.8/packages/coverage-v8) Updates `eslint` from 9.38.0 to 9.39.1 - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](https://github.com/eslint/eslint/compare/v9.38.0...v9.39.1) Updates `globals` from 16.4.0 to 16.5.0 - [Release notes](https://github.com/sindresorhus/globals/releases) - [Commits](https://github.com/sindresorhus/globals/compare/v16.4.0...v16.5.0) Updates `happy-dom` from 20.0.8 to 20.0.10 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](https://github.com/capricorn86/happy-dom/compare/v20.0.8...v20.0.10) Updates `typescript` from 5.8.3 to 5.9.3 - [Release notes](https://github.com/microsoft/TypeScript/releases) - [Changelog](https://github.com/microsoft/TypeScript/blob/main/azure-pipelines.release-publish.yml) - [Commits](https://github.com/microsoft/TypeScript/compare/v5.8.3...v5.9.3) Updates `typescript-eslint` from 8.46.2 to 8.46.4 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.46.4/packages/typescript-eslint) Updates `vite` from 7.1.12 to 7.2.2 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v7.2.2/packages/vite) Updates `vitest` from 4.0.3 to 4.0.8 - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.0.8/packages/vitest) --- updated-dependencies: - dependency-name: "@chakra-ui/react" dependency-version: 3.29.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: "@tanstack/react-query" dependency-version: 5.90.8 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: axios dependency-version: 1.13.2 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: react-router-dom dependency-version: 7.9.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: "@eslint/compat" dependency-version: 1.4.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: "@eslint/js" dependency-version: 9.39.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: "@trivago/prettier-plugin-sort-imports" dependency-version: 6.0.0 dependency-type: direct:development update-type: version-update:semver-major dependency-group: edge-ui-package-updates - dependency-name: "@types/node" dependency-version: 24.10.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: "@types/react" dependency-version: 19.2.4 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: "@types/react-dom" dependency-version: 19.2.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: "@vitejs/plugin-react-swc" dependency-version: 4.2.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: "@vitest/coverage-v8" dependency-version: 4.0.8 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: eslint dependency-version: 9.39.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: globals dependency-version: 16.5.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: happy-dom dependency-version: 20.0.10 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: typescript dependency-version: 5.9.3 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: typescript-eslint dependency-version: 8.46.4 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates - dependency-name: vite dependency-version: 7.2.2 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: edge-ui-package-updates - dependency-name: vitest dependency-version: 4.0.8 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: edge-ui-package-updates ... Signed-off-by: dependabot[bot] <[email protected]> * Downgrade typescript which can not be upgraded --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jens Scheffler <[email protected]> * feat: async apprise notifier (#57541) * Mask kwargs on illegal args (#58252) * Remove @pytest.mark.xfail and fix incorrect assertion - Remove @pytest.mark.xfail decorator as the test is now fixed - Remove incorrect assertion checking mock_manager.read_pod_logs which is never called by _write_logs() (it calls client.read_namespaced_pod_log directly instead) Both parameterized test cases now pass: - test_async_write_logs_should_execute_successfully[True] ✅ - test_async_write_logs_should_execute_successfully[False] ✅ Addresses reviewer feedback to remove xfail flag as proof the test works. * Add poll_interval attribute to HttpEventTrigger (#57583) * Add poll_interval attribute to HttpEventTrigger * Add info on poll_interval to HttpEventTrigger docs * Add info on poll_interval to HttpEventTrigger docs * Add poll_interval attribute to tests * Update providers/http/docs/triggers.rst Co-authored-by: Ryan Hatter <[email protected]> --------- Co-authored-by: Ryan Hatter <[email protected]> * Add missing test for amazon/aws/executors/ecs/test_utils.py (#58139) * Create test_utils.py * remove * Update test_utils.py * Update test_utils.py --------- Co-authored-by: zk-zou <[email protected]> * Apply ruff formatting fixes - Remove unused imports: BytesIO and HTTPResponse - Format function parameters on multiple lines for better readability * Add Kerberos dependency to hive provider (#55773) * CI: Upgrade ruff version in CI environment (#58287) * Update documentation for providers 14 Nov 2025 (#58284) * HookLevelLineage can now add arbitrary data with add_extra (#57620) It's useful, but it's missing an option to attach non-asset related metadata collected from hooks (f.e. query_id, or query_text), that can be also useful in context of lineage and desired by hook lineage readers. For example, having the executed SQL query enables SQL parsing on the reader side, retrieving input and output datasets, which can significantly improve lineage extraction accuracy. * Bump min version of openlineage libraries to 1.40.0 to fix compat issues (#58302) * Add logging to test_example_dag_hashes_are_always_consistent (#58264) Add logging to test_example_dag_hashes_are_always_consistent * Allow virtualenv code to access connections/variables and send logs (#58148) (This is the second attempt at this PR) This change is quite simple, as it simply calls the function added in #57212 and #58147 if it's available. Up until the point that it is released this code will not hit the getattr will return None and be a no-op. I didn't use the walrus operator here as theoretically someone could be using this to run in a Python that doesn't support it. Fixes #51422, fixes #54706. This doesn't have any unit tests, as a) it's relatively simple, and b) due to how we run in the same process in the tests, there is no socket to reconnect to, so we can't actually exercise this code * Drop version handling during config write as its not a practical case (#58269) * Making `conf.as_dict` extendable by plugging in config sources (#58268) * Delete dag_doc.md from the git tree which is a wrongly committed file (#58309) * Remove SDK reference for NOTSET in Airflow Core (#58258) * Display static asset metadata (extra) in Asset details page (#57710) * Display static asset metadata (extra) in Asset details page * Add i18n translation for static Asset Metadata and minor UI cleanup * Add i18n translation key 'ststic_asset_metadata' to all locale asset.jason files * Rename Static Asset Metadata to Additional Data * Properly highlight TaskGroup nodes when selected (#58118) * Fix link on installing-from-sources page (#58323) * fix static checks in main (#58325) * Fix AwaitMessageSensor to accept timeout and soft_fail parameters (#57863) (#58070) - Changed AwaitMessageSensor and AwaitMessageTriggerFunctionSensor to inherit from BaseSensorOperator instead of BaseOperator - This enables both sensors to accept standard sensor parameters like timeout and soft_fail - Added comprehensive test coverage for timeout and soft_fail parameters - Updated documentation to reflect the new parameters Fixes #57863 * Fix: TriggerDagRunOperator stuck in deferred state with reset_dag_run (#57756) (#57968) When TriggerDagRunOperator is used with deferrable=True, wait_for_completion=True, reset_dag_run=True, and a fixed trigger_run_id, the operator becomes permanently stuck in deferred state after clearing and re-running. Root cause: When reset_dag_run=True is used with a fixed run_id, the database preserves the original logical_date from the first run. However, on subsequent runs after clearing, the operator calculates a NEW logical_date based on the current time. The DagStateTrigger was being created with this newly calculated logical_date, causing a mismatch when querying the database - the trigger looked for a DAG run with the new logical_date but the database contained the original logical_date, causing the query to return zero results indefinitely. Solution: - Modified _handle_trigger_dag_run() in task_runner.py to pass execution_dates=None to DagStateTrigger when run_ids is provided, since run_id alone is sufficient and globally unique - Added test test_handle_trigger_dag_run_deferred_with_reset_uses_run_id_only to verify the fix and prevent regression The fix ensures that both deferrable and non-deferrable modes use identical logic for determining DAG run completion - querying by run_id and state only, without filtering by logical_date which can become stale when resets are involved. * Updates to release process of providers (#58316) * Updates to release process of providers There are several updates to the release process in preparation for rotational release management process: * documentation has been simplified and unnecessary steps and digressions are move out of the way * some commands had been simplified, optional steps have been moved at the end of the doc * commented out PRs in release notes are automatically removed from the Issue created for users * better guideliens on how to iterate with the issue by removing unneeded PRs * gh issue create used instead of manually hand-crafting the URL is used to trigger issue creation * github token retrieval is simplified for prepare-issues command * stale ".. Please review " comments were removed from the changelogs * We are using now ${RELEASE_DATE} subfolder in SVN to store the providers - this will allow several parallel releases to be run. * Update providers/amazon/docs/changelog.rst Co-authored-by: Bugra Ozturk <[email protected]> --------- Co-authored-by: Bugra Ozturk <[email protected]> * Doc: update chart info about built-in secrets and environment variables (#58317) * Fix group TI tab memory (#58288) * fix: Prevent preserving tabs when navigating to Task Groups * refactor: Centralize keyboard navigation logic to use * test: update test, Groups should not preserve tabs * Breeze: Automatically set CHOKIDAR_USEPOLLING for WSL users in --dev-mode (#57846) (#58080) * Docs: Add note for WSL users about CHOKIDAR_USEPOLLING for UI hot reloading (#57846) * Breeze: Automatically set CHOKIDAR_USEPOLLING for WSL users in --dev-mode (#57846) * Breeze: Automatically set CHOKIDAR_USEPOLLING for WSL users in --dev-mode (#57846) * Update dev/breeze/src/airflow_breeze/commands/developer_commands.py Co-authored-by: LIU ZHE YOU <[email protected]> --------- Co-authored-by: LIU ZHE YOU <[email protected]> * Update main as 3.1.3 has been released (#58341) * Add JSON serialization for kubeconfig in AsyncKubernetesHook (#57169) * Add JSON serialization for kubeconfig in AsyncKubernetesHook * Add debug logging for kubeconfig serialization in AsyncKubernetesHook * Ran perk --------- Co-authored-by: Kalyan R <[email protected]> * Chart: Default airflow version to 3.1.3 (#58343) * Simolify and modernize provider documentation preparation instructions (#58306) The instructions were partially outdated and quite a bit too chatty, they did not focus on actual tasks to do but also contained some digressions and more detailed description of some provider principles. This change makes it simpler to follow the process by the release manager, moves the docuentation of details to PROVIDERS.rst and removes largely duplicated PROVIDER_DISTRIBUTION_DETAILS.md The "--incremental-update" flag is specifically added and mentioned in the doc on what should be done when new commits are added after initial release notes have been prepared, guiding the release manager more explicitly what to do and providing instructions. * Increase waiter delay for ecs run tasks in system tests (#58338) The default is 6s which is quite aggressive and causes throttling issues in the test harness. Tests don't need that kind of fidelity and can wait much longer between attempts to fire the trigger. --------- Signed-off-by: Xch1 <[email protected]> Signed-off-by: rich7420 <[email protected]> Signed-off-by: Anusha Kovi <[email protected]> Signed-off-by: Your Name <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Anusha Kovi <[email protected]> Co-authored-by: Kalyan R <[email protected]> Co-authored-by: Ephraim Anierobi <[email protected]> Co-authored-by: Xch1 <[email protected]> Co-authored-by: Vincent <[email protected]> Co-authored-by: David Blain <[email protected]> Co-authored-by: Tzu-ping Chung <[email protected]> Co-authored-by: Pierre Jeambrun <[email protected]> Co-authored-by: Amogh Desai <[email protected]> Co-authored-by: Stefan Wang <[email protected]> Co-authored-by: ChenChen Lai <[email protected]> Co-authored-by: Yun-Ting Chiu <[email protected]> Co-authored-by: Sebastian Daum <[email protected]> Co-authored-by: GUAN-HAO HUANG <[email protected]> Co-authored-by: Ankit Chaurasia <[email protected]> Co-authored-by: Kacper Muda <[email protected]> Co-authored-by: Niko Oliveira <[email protected]> Co-authored-by: kyounghoonJang <[email protected]> Co-authored-by: Henry Chen <[email protected]> Co-authored-by: Jens Scheffler <[email protected]> Co-authored-by: Ash Berlin-Taylor <[email protected]> Co-authored-by: Jarek Potiuk <[email protected]> Co-authored-by: Daniel Standish <[email protected]> Co-authored-by: AutomationDev85 <[email protected]> Co-authored-by: Wei Lee <[email protected]> Co-authored-by: Bugra Ozturk <[email protected]> Co-authored-by: LI,JHE-CHEN <[email protected]> Co-authored-by: Yang Yule <[email protected]> Co-authored-by: Gurram Sai Ganesh <[email protected]> Co-authored-by: Shahar Epstein <[email protected]> Co-authored-by: Lê Nam Khánh <[email protected]> Co-authored-by: Simon Meng <[email protected]> Co-authored-by: Elad Kalif <[email protected]> Co-authored-by: Aaron Wolmutt <[email protected]> Co-authored-by: Guan-Ming (Wesley) Chiu <[email protected]> Co-authored-by: majorosdonat <[email protected]> Co-authored-by: Aaron Chen <[email protected]> Co-authored-by: Steve Ahn <[email protected]> Co-authored-by: Ramit Kataria <[email protected]> Co-authored-by: Jorge Rocamora <[email protected]> Co-authored-by: Your Name <[email protected]> Co-authored-by: Srabasti Banerjee <[email protected]> Co-authored-by: Aldo <[email protected]> Co-authored-by: ecodina <[email protected]> Co-authored-by: LIU ZHE YOU <[email protected]> Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Prafful Javare <[email protected]> Co-authored-by: Aritra Basu <[email protected]> Co-authored-by: Shubham Raj <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: ha2hi <[email protected]> Co-authored-by: Dheeraj Turaga <[email protected]> Co-authored-by: Hajun Yoo <[email protected]> Co-authored-by: Yeonguk Choo <[email protected]> Co-authored-by: Juarez Rudsatz <[email protected]> Co-authored-by: VladaZakharova <[email protected]> Co-authored-by: Nikita Kalganov <[email protected]> Co-authored-by: Nikita <[email protected]> Co-authored-by: Nikita.Kalganov <[email protected]> Co-authored-by: Vasu Madaan <[email protected]> Co-authored-by: Dev-iL <[email protected]> Co-authored-by: Nathan Hadfield <[email protected]> Co-authored-by: Jake Roach <[email protected]> Co-authored-by: darkag <[email protected]> Co-authored-by: Arkadiusz Bach <[email protected]> Co-authored-by: Mikhail Ilchenko <[email protected]> Co-authored-by: Jens Scheffler <[email protected]> Co-authored-by: Karen Braganza <[email protected]> Co-authored-by: Ryan Hatter <258…
ephraimbuddy
pushed a commit
that referenced
this pull request
Dec 3, 2025
…58169) (cherry picked from commit 3afad4d) Co-authored-by: Shubham Raj <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue -
Fix -
Screen.Recording.2025-11-08.at.5.06.57.AM.mov
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.