Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

- Add a flag to determine if database initialization steps should be executed ([#669]).
- Add new roles for dag-processor and triggerer processes ([#679]).
- Added a note on webserver workers to the trouble-shooting section ([#685]).

### Fixed

Expand All @@ -22,6 +23,7 @@
[#678]: https://github.com/stackabletech/airflow-operator/pull/678
[#679]: https://github.com/stackabletech/airflow-operator/pull/679
[#683]: https://github.com/stackabletech/airflow-operator/pull/683
[#685]: https://github.com/stackabletech/airflow-operator/pull/685

## [25.7.0] - 2025-07-23

Expand Down
26 changes: 26 additions & 0 deletions docs/modules/airflow/pages/troubleshooting/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,29 @@ webservers: # Also add to other roles!
----

See e.g. https://github.com/minio/minio/issues/20845[this MinIO issue] for details.

== Setting API Workers

In Airflow the webserver (called the API Server in Airflow 3.x+) can use multiple workers.
This is determined by the environment variable `+AIRFLOW__API__WORKERS+` and is set by default to `4` in Airflow 2.x and `1` in Airflow 3.x+.
The reason for this difference is that Airflow uses a backend library to manage child processes and in 3.x+ this library can cause child processes to be killed if a hard-coded startup timeout is exceeded.
For most cases with Airflow 3.x+ a default of `1` should be sufficient, but if you run into performance issues and would like to add more workers, you can either change the environment variable using `envOverrides`:

[source,yaml]
----
webservers:
envOverrides:
AIRFLOW__API__WORKERS: 2 # something other than the default of 1
----

or modulate multiple worker processes at the level of webserver, keeping the default of a single worker per webserver:

[source,yaml]
----
webservers:
roleGroups:
default:
replicas: 2 # add a replica (with a single worker)
----

TIP: Our recommendation is to increase the webserver replicas, with each webserver running a single worker, as this removes the risk of running into timeouts or memory issues.
Loading