Skip to content

fix(mongodb): serve :of-type, reference :identifier, :[type] and :above/:below - #1419

Merged
smunini merged 3 commits into
mainfrom
fix/1408-mongodb-search-modifiers
Sep 22, 2026
Merged

smunini merged 3 commits into
mainfrom
fix/1408-mongodb-search-modifiers

Conversation

@smunini

@smunini smunini commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Summary

MongoDB refused search modifiers that SQLite, PostgreSQL and Elasticsearch all serve, so the same request succeeded or failed depending on HFS_STORAGE_BACKEND.

These were reachable over plain HTTP

These results come from a real hfs on MongoDB, with main's MongoDB code before the fix and this branch after.

Request Before After
Observation?subject:Patient=p1 400 unsupported modifier Patient 200, the Observation whose subject is Patient/p1
Observation?subject:Group=p1 400 200, the Observation whose subject is Group/p1
Observation?subject:identifier=http://example.org/mrn|12345 400 200, correct match
Patient?identifier:of-type=…v2-0203|MR|12345 400 unsupported modifier ofType 200, correct match
Observation?subject:below=Patient 400 200, correct match
Chained (subject:Patient.identifier=…) and _has worked unchanged
  • REST passes :[type] to the backend as a modifier, not as a type-qualified value, so the refusal was reachable over HTTP.
  • The chained form already worked, because REST resolves chains before the backend sees them.

Why the Inferno MongoDB legs stayed green: the US Core test kit never sends these forms.

  • Its "reference with type" search puts the type in the value (patient=Patient/<id>).
  • It sends no :identifier or :of-type.
  • A real client that sends subject:Patient=… was refused on MongoDB.

Root cause

The builders were never written, so the modifiers fell into the UnsupportedModifier catch-all. Reference :above/:below were refused one step earlier, in validate_query_support.

Nothing was missing from the index. The writer has always stored the identifier type system and code on identifier rows, and a partial index for them already exists. No reindex is needed.

Fix

  • :of-type
    • It compares the type system, the type code and the value.
    • An empty part is not compared, as on SQLite and PostgreSQL.
    • Anything other than three parts matches nothing. The condition is never dropped.
  • :[type]
    • subject:Patient=1 builds the same filter as subject=Patient/1, so it cannot match Group/1.
    • Qualified reference values are now version-agnostic (strip_reference_version), as on every other backend.
  • :identifier
    • It has the SQLite and PostgreSQL meaning: the reference's target carries the identifier.
    • A MongoDB filter document cannot join, so the targets are resolved first, then turned into one index-bounded $in. The lookup is tenant-scoped and bounded to the parameter's declared target types.
    • More than 10,000 targets is refused with TooManyResults. The result is never truncated.
  • Reference :above/:below reuse the uri shapes.
  • modifiers_for_type now advertises what is actually served.

New test suite

New cross-backend suite: crates/persistence/tests/search/modifier_parity_suite.rs.

  • It walks every modifier that SearchModifier::is_valid_for allows, on every parameter type, on all four backends.
  • Each backend's known differences are stated explicitly in the suite, so a backend cannot silently fall behind again.
  • It also covers AND-combination, search_count and tenant isolation.

Verification

Run after merging current main, on real containers:

  • cargo test -p helios-persistence --lib --features sqlite,postgres,mongodb,elasticsearch: 1782 passed
  • mongodb_tests (full): 206 passed
  • postgres_tests (full): 268 passed
  • elasticsearch_tests (full): 141 passed
  • sqlite_tests + search_suite: 219 passed
  • cargo +1.98.1 clippy -p helios-persistence --features R4,R4B,R5,R6,sqlite,postgres,mongodb,elasticsearch,s3 --all-targets with the CI flags: clean. This compiles code gated on several FHIR versions.
  • cargo fmt --check: clean

Not run on this branch alone:

  • helios-rest tests. No REST code is touched.
  • The full --workspace --all-features compile. That runs on the combined tree of the open PRs.

Fixes #1408

…:below

MongoDB refused three standard search modifiers that SQLite, PostgreSQL
and Elasticsearch serve, so the same request succeeded or failed with
HFS_STORAGE_BACKEND: token `:of-type`, reference `:identifier` and the
reference `:[type]` qualifier all fell into the `UnsupportedModifier`
catch-all of their filter builder. Reference `:above`/`:below` were
refused one step earlier, by `validate_query_support`.

Root cause: the builders were never written. Nothing was missing from
the index - the writer has always stored `value_identifier_type_system`
/ `value_identifier_type_code` on identifier rows (and the partial index
`idx_search_identifier_type_v2` exists for them), so no reindex is
needed.

- `:of-type` compares type system, type code and value; an empty part is
  not compared (as SQLite/PostgreSQL); anything but three parts matches
  nothing rather than dropping the condition.
- `:[type]` is the qualified reference: `subject:Patient=1` builds the
  very filter `subject=Patient/1` builds, so it cannot match `Group/1`. A
  value naming another type matches nothing. Qualified reference values
  are now version-agnostic (`strip_reference_version`), like every other
  backend.
- `:identifier` has the SQLite/PostgreSQL meaning (the reference's target
  carries the identifier). A filter document cannot join, so
  `matching_resource_ids` resolves the targets first - tenant-scoped, and
  bounded to the parameter's declared target types so the token index
  serves it - and turns the parameter into one index-bounded `$in` of
  `Type/id` plus an anchored `_history` regex each. More than 10 000
  targets is refused (`TooManyResults`), never truncated.
- reference `:above`/`:below` reuse the uri shapes.

`modifiers_for_type` advertises what is now served, and the new
cross-backend `modifier_parity_suite` walks every modifier
`SearchModifier::is_valid_for` allows on every parameter type on all four
backends, with each backend's known differences stated explicitly so no
backend can silently fall behind again.

Fixes #1408
… tenants

The suite's `:identifier` lookup is a second query; a cell now proves it is
tenant-scoped, three cells AND a modified parameter with a plain one so it
is also exercised as the non-driving filter, and every cell checks that
`search_count` counts what `search` returns.

Refs #1408
smunini added a commit that referenced this pull request Sep 21, 2026
…omic-versioned-writes

#1419 and this PR each add a backend-agnostic suite (modifier_parity_suite
there, versioned_write_race_suite here) and wire it into the same spots of
tests/mongodb_tests.rs and tests/postgres_tests.rs, so whichever merges
second conflicts. Resolved here, making the order #1419 then this PR; every
test kept unchanged. The resulting tree is byte-identical to the
corresponding step of a scratch merge of main with all four open PRs
(#1419, #1421, #1422, #1423), which compiled under
`cargo test --workspace --all-features --no-run` and passed the full SQLite,
PostgreSQL, MongoDB, Elasticsearch and REST suites.
smunini added a commit that referenced this pull request Sep 21, 2026
…nto feat/1406-shared-conditional-patch

#1419, #1421 and this PR each add a backend-agnostic suite and wire it into
the same spots of tests/mongodb_tests.rs, tests/postgres_tests.rs and
tests/sqlite_tests.rs, so whichever merges later conflicts. Resolved here,
making the order #1419, #1421, then this PR; every test kept unchanged. The
resulting tree is byte-identical to the corresponding step of a scratch merge
of main with all four open PRs (#1419, #1421, #1422, #1423), which compiled
under `cargo test --workspace --all-features --no-run` and passed the full
SQLite, PostgreSQL, MongoDB, Elasticsearch and REST suites.
@codecov

codecov Bot commented Sep 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.44789% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...es/persistence/src/backends/mongodb/search_impl.rs 98.60% 6 Missing ⚠️
crates/persistence/src/backends/mongodb/backend.rs 95.45% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@smunini
smunini merged commit b36659d into main Sep 22, 2026
35 of 36 checks passed
@smunini
smunini deleted the fix/1408-mongodb-search-modifiers branch September 22, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MongoDB rejects :of-type, reference :identifier and reference :[type] that the other backends support

1 participant