Skip to content

Fix task state store Execution API rejecting keys that contain slashes - #69178

Merged
vatsrahul1001 merged 5 commits into
apache:mainfrom
takayoshi-makabe:fix-execution-api-task-state-store-key-path
Aug 3, 2026
Merged

Fix task state store Execution API rejecting keys that contain slashes#69178
vatsrahul1001 merged 5 commits into
apache:mainfrom
takayoshi-makabe:fix-execution-api-task-state-store-key-path

Conversation

@takayoshi-makabe

@takayoshi-makabe takayoshi-makabe commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Keys containing / (e.g. "spark/job_id") silently fail with 404 when accessed through the Execution API used by running tasks. The routes were defined with {key} instead of {key:path}, so FastAPI treated the slash as a path separator and could not match the route.

The Public API (used by UI/external clients), defined in airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state_store.py, already uses /{key:path} and is correct. This PR aligns the Execution API (airflow-core/src/airflow/api_fastapi/execution_api/routes/task_state_store.py) with that existing pattern.

A task managing multiple external jobs may naturally use slash-namespaced keys such as "spark/job_id" or "bigquery/status" to group related state — these silently returned 404 before this fix.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Claude Code


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@SameerMesiah97 SameerMesiah97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but CI needs to be triggered.

@kaxil kaxil added this to the Airflow 3.3.0 milestone Jun 30, 2026
@kaxil kaxil added the backport-to-v3-3-test Backport to v3-3-test label Jun 30, 2026

@ashb ashb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we accept path, or should we make the client url encode the key it sends? (Or both?)

@takayoshi-makabe

Copy link
Copy Markdown
Contributor Author

@ashb
{key:path} seems like the right call here — the Public API already uses the same pattern, which keeps the two APIs consistent.

Client-side encoding could also be a good idea for added safety, but I think that can be treated as a separate concern.

@amoghrajesh

Copy link
Copy Markdown
Contributor

@ashb I think both is a good idea. {key:path} on the server is the right defensive fix in any case, it matches what the core API already does and handles clients that do not encode - task sdk might not be the only client in the future.

@takayoshi-makabe Worth including in this PR since it's the same root cause.

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@takayoshi-makabe changes look fine, can you handle client side encoding too? I will approve once thats done

@takayoshi-makabe

Copy link
Copy Markdown
Contributor Author

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the tests as suggested.

Comment thread task-sdk/tests/task_sdk/api/test_client.py Outdated
@takayoshi-makabe

Copy link
Copy Markdown
Contributor Author

@amoghrajesh
I have simplified the test. Could you please review it again?

@takayoshi-makabe

Copy link
Copy Markdown
Contributor Author

@amoghrajesh Following up — addressed the test feedback on 2026-07-01, CI's green. Could you take another look when you have a chance?

@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jul 18, 2026

@vatsrahul1001 vatsrahul1001 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Good to merge post review from code owners(@ashb @amoghrajesh @kaxil )

@vatsrahul1001
vatsrahul1001 merged commit 4899e90 into apache:main Aug 3, 2026
101 of 102 checks passed
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 3, 2026
…ntain slashes (apache#69178)

* Fix task state store API key path

* Revert uv.lock

* Add client side

* Simplified the test
(cherry picked from commit 4899e90)

Co-authored-by: Takayoshi Makabe <48040599+takayoshi-makabe@users.noreply.github.com>
aws-airflow-bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 3, 2026
…ntain slashes (apache#69178)

* Fix task state store API key path

* Revert uv.lock

* Add client side

* Simplified the test
(cherry picked from commit 4899e90)

Co-authored-by: Takayoshi Makabe <48040599+takayoshi-makabe@users.noreply.github.com>
vatsrahul1001 pushed a commit that referenced this pull request Aug 4, 2026
…ntain slashes (#69178) (#70967)

* Fix task state store API key path

* Revert uv.lock

* Add client side

* Simplified the test
(cherry picked from commit 4899e90)

Co-authored-by: Takayoshi Makabe <48040599+takayoshi-makabe@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v3-3-test Backport to v3-3-test ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants