Skip to content

Resolve backfill_id in the access dependency with the type the routes declare - #70889

Merged
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:resolve-backfill-id-with-the-routes-declared-type
Aug 4, 2026
Merged

Resolve backfill_id in the access dependency with the type the routes declare#70889
vatsrahul1001 merged 3 commits into
apache:mainfrom
potiuk:resolve-backfill-id-with-the-routes-declared-type

Conversation

@potiuk

@potiuk potiuk commented Aug 1, 2026

Copy link
Copy Markdown
Member

The backfill routes declare backfill_id: NonNegativeInt, but
requires_access_backfill parsed the raw path value with int() and swallowed
the failure. Those two parsers do not agree.

The divergence

supplied backfill_id NonNegativeInt (the handler) int() (the dependency)
"42" 42 42
"42.0" 42 ValueError
"42.00" 42 ValueError
"42.", "42e0", "42.5", "0x2a" rejected ValueError

pydantic's lax mode accepts the two decimal spellings and coerces them to an
int; int() rejects them.

FastAPI resolves route dependencies before the endpoint's own parameter
validation, so for those spellings the dependency's parse failed, it left the
Dag unresolved, and the request went on to be served by the handler against
backfill 42. The dependency and the handler disagreed about which backfill --
and therefore which Dag -- the request concerned.

Approach

Parse with the same TypeAdapter(NonNegativeInt) the routes declare, so the
two cannot diverge by construction. The adapter is module-level and named, so
the coupling to the route signature is explicit rather than re-derived.

This deliberately does not introduce a stricter parser. An earlier revision
used int() with an explicit 400, which changed the public API contract: a
malformed path value such as /backfills/invalid_id previously produced
FastAPI's structured 422 (loc: ["path", "backfill_id"]), and a 400 from
the dependency preempted it. Five existing route tests caught that. Matching
the declared type keeps every existing status code and body shape intact --
values both parsers reject still yield the handler's 422, and an unknown
backfill still yields each route's own 404 wording.

Test plan

  • test_requires_access_backfill_authorizes_the_backfill_the_handler_will_act_on
    -- parametrized over 42, 42.0, 42.00; asserts the Dag comes from the
    resolved backfill and not from a value supplied on the request
  • Verified against unmodified code: 42.0 and 42.00 fail, 42 passes --
    so the test pins the divergence and not the plumbing
  • airflow-core/tests/unit/api_fastapi/core_api/test_security.py -- 111 passed
  • airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_backfills.py
    -- 63 passed, including the four test_invalid_id 422 cases and test_no_exist
  • ruff check / ruff format clean
Was generative AI tooling used to co-author this PR?
  • Yes — Claude Opus 5 (1M context)

Generated-by: Claude Opus 5 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

@potiuk potiuk added this to the Airflow 3.3.1 milestone Aug 1, 2026
@potiuk potiuk added the backport-to-v3-3-test Backport to v3-3-test label Aug 1, 2026
@boring-cyborg boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Aug 1, 2026

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can do better here, maybe @pierrejeambrun has an opinion too.

Comment thread airflow-core/tests/unit/api_fastapi/core_api/test_security.py Outdated
Comment thread airflow-core/src/airflow/api_fastapi/core_api/security.py
potiuk added 2 commits August 3, 2026 21:44
… declare

The backfill routes declare `backfill_id: NonNegativeInt`, but
`requires_access_backfill` parsed the raw path value with `int()` and
swallowed the failure. The two parsers do not agree: pydantic's lax mode
validates "1.0" and "1.00" to 1, while `int()` rejects both.

Dependencies resolve before the endpoint's own parameter validation, so for
those spellings the dependency left the Dag unresolved on a request the
handler then served against backfill 1 -- the two disagreed about which Dag
the request concerned.

Parse with the same TypeAdapter the routes declare so they cannot diverge.
An unspecced Mock accepts any attribute, so the test would keep passing if the
dependency started reading something the real Request, Session or Backfill does
not have.
@potiuk
potiuk force-pushed the resolve-backfill-id-with-the-routes-declared-type branch from 00f6305 to 8f271a5 Compare August 3, 2026 19:56
@potiuk

potiuk commented Aug 3, 2026

Copy link
Copy Markdown
Member Author

cc: @pierrejeambrun ?

@potiuk

potiuk commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Both threads addressed and resolved. Mocks now carry spec.

I did not move backfill_id into the dependency signature: the dependency is also attached to GET /backfills, POST /backfills and POST /backfills/dry_run, which have no backfill_id path parameter, so FastAPI would resolve it as a query parameter there — 422-ing those routes if declared bare, or exposing ?backfill_id= in the public schema and letting a caller steer authorization at a backfill the handler never touches. Detail in the thread; a shared type alias is the alternative if drift is the concern.

@amoghrajesh @pierrejeambrun


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

@potiuk
potiuk requested a review from amoghrajesh August 4, 2026 00:48

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, seems like a valid issue to me. Can someone with better backfill API knowledge look at this one to prevent any ripple effects?

@ephraimbuddy ephraimbuddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Just nits

Comment thread airflow-core/tests/unit/api_fastapi/core_api/test_security.py Outdated
Comment thread airflow-core/src/airflow/api_fastapi/core_api/security.py Outdated
A backfill_id that parses but matches no row falls through to the body's dag_id,
so an unknown backfill answers 404 where an unauthorized one answers 403 and a
caller can tell which ids exist. That is a separate fix from the parser
divergence this change closes, and it has to keep the three body-authorized
routes working, so it is tracked rather than folded in here.

The comment above the adapter also loses the history that led to it; what
matters going forward is the rule it states.
@vatsrahul1001 vatsrahul1001 added the type:bug-fix Changelog: Bug Fixes label Aug 4, 2026
@vatsrahul1001
vatsrahul1001 merged commit a6265b7 into apache:main Aug 4, 2026
79 checks passed
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test PR Link

vatsrahul1001 added a commit that referenced this pull request Aug 4, 2026
…e the routes declare (#70889) (#71090)

* Resolve backfill_id in the access dependency with the type the routes declare

The backfill routes declare `backfill_id: NonNegativeInt`, but
`requires_access_backfill` parsed the raw path value with `int()` and
swallowed the failure. The two parsers do not agree: pydantic's lax mode
validates "1.0" and "1.00" to 1, while `int()` rejects both.

Dependencies resolve before the endpoint's own parameter validation, so for
those spellings the dependency left the Dag unresolved on a request the
handler then served against backfill 1 -- the two disagreed about which Dag
the request concerned.

Parse with the same TypeAdapter the routes declare so they cannot diverge.

* Use spec'd mocks in the backfill authorization dependency test

An unspecced Mock accepts any attribute, so the test would keep passing if the
dependency started reading something the real Request, Session or Backfill does
not have.

* Point at the tracking issue for the unknown-backfill fallback

A backfill_id that parses but matches no row falls through to the body's dag_id,
so an unknown backfill answers 404 where an unauthorized one answers 403 and a
caller can tell which ids exist. That is a separate fix from the parser
divergence this change closes, and it has to keep the three body-authorized
routes working, so it is tracked rather than folded in here.

The comment above the adapter also loses the history that led to it; what
matters going forward is the rule it states.
(cherry picked from commit a6265b7)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API backport-to-v3-3-test Backport to v3-3-test type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants