From 2e99fe7e7c18587c8b7ef8d0ad593727255aacb0 Mon Sep 17 00:00:00 2001 From: Taleef Date: Tue, 8 Sep 2026 13:25:27 -0400 Subject: [PATCH 1/6] feat(measure): the Maui sandbox routes the ACO's whole computable set, and a patient outside a measure's population is not a case (ADR-078) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Owner decision 2026-09-08: every measure the pilot group sent us, working and visible on the sandbox. WORKWELL_OFFICIAL_MEASURES on both Maui workflows is now cms122,cms125,cms2,cms130,cms165,cms137, and the deploy's vendor step adds CMS137, the one pilot measure the build did not vendor. Evidence: cms2's flip gate over all 20,000 corpus patients (36/36 MADiE, 17,795 in the initial population, 5,413 actionable, 0 errors — docs/evidence/FLIP_GATE_2026-09-08_CMS2.md); cms137's banked gate; the cross-engine sweeps. cms130 and cms165 pin VSAC-completed sidecars that do not resolve locally, so a manual flip-gate.yml workflow now vendors with the secret, sweeps the roster and uploads the report. Routing six measures made ADR-043's recorded fan-out the worklist: an official outcome outside the initial population persisted as MISSING_DATA and opened a MEDIUM case, so 91–97 % of the sandbox's open cases were patients the measure does not concern. The pipeline now reads the executor's own inInitialPopulation:false and treats it as close-only — no case where none exists, an active one closed by the system under OUT_OF_POPULATION, audited, reopenable — and the CDS card surface applies the same rule off the persisted evidence. The authored engine never sets the flag, so an authored MISSING_DATA still opens a case. The two conditions the locked decision named — 305 surviving the final rule for cms137, profile-stamped real blood pressures for cms165 — now gate the PHI phase rather than the sandbox flip (LOCKED_DECISIONS §4A.2 SINCE). --- .github/workflows/deploy-maui-mieweb.yml | 7 ++- .github/workflows/reconcile-maui-mieweb.yml | 2 +- CLAUDE.md | 14 ++--- backend-ts/src/case/case-logic.test.ts | 23 ++++++++ backend-ts/src/case/case-logic.ts | 21 +++++++- backend-ts/src/cds/cards.test.ts | 12 +++++ backend-ts/src/cds/cards.ts | 8 ++- backend-ts/src/run/run-pipeline.test.ts | 45 ++++++++++++++++ backend-ts/src/run/run-pipeline.ts | 26 +++++++-- backend-ts/src/stores/case-store.ts | 6 +++ .../stores/postgres/case-store-postgres.ts | 4 +- .../src/stores/sqlite/case-store-sqlite.ts | 3 ++ docs/ADR_INDEX.md | 5 +- docs/DATA_MODEL_CONTRACTS.md | 6 +++ docs/DECISIONS.md | 54 +++++++++++++++++++ docs/DEPLOY.md | 13 +++-- docs/JOURNAL.md | 33 ++++++++++++ docs/LOCKED_DECISIONS.md | 8 +++ docs/MEASURES.md | 22 ++++---- docs/evidence/FLIP_GATE_2026-09-08_CMS2.md | 31 +++++++++++ 20 files changed, 311 insertions(+), 32 deletions(-) create mode 100644 docs/evidence/FLIP_GATE_2026-09-08_CMS2.md diff --git a/.github/workflows/deploy-maui-mieweb.yml b/.github/workflows/deploy-maui-mieweb.yml index 2e458911..309b285e 100644 --- a/.github/workflows/deploy-maui-mieweb.yml +++ b/.github/workflows/deploy-maui-mieweb.yml @@ -111,6 +111,7 @@ jobs: node scripts/vendor-official-measure.mjs --measure CMS138FHIRTobaccoScrnCessation --catalog-id cms138 --strip-elm-annotations --complete-terminology node scripts/vendor-official-measure.mjs --measure CMS130FHIRColorectalCancerScrn --catalog-id cms130 --strip-elm-annotations --complete-terminology node scripts/vendor-official-measure.mjs --measure CMS165FHIRControllingHighBP --catalog-id cms165 --strip-elm-annotations --complete-terminology + node scripts/vendor-official-measure.mjs --measure CMS137FHIRSUDTxInitEngagement --catalog-id cms137 --strip-elm-annotations --complete-terminology # The vendored artifact must not have changed — only the gitignored sidecar is new. A non-empty # diff means the committed bundle does not match its pin, which must never reach a deploy image. - name: The committed artifact is reproducible from its pin @@ -276,7 +277,11 @@ jobs: {key: "WORKWELL_INSTANCE", value: "maui"}, {key: "WORKWELL_SCHEDULER_ENABLED", value: "true"}, {key: "WORKWELL_VSAC_API_KEY", value: $vsac_api_key}, - {key: "WORKWELL_OFFICIAL_MEASURES", value: "cms122,cms125"}, + # THE FLIP (ADR-045; ADR-078, owner decision 2026-09-08): the sandbox routes the ACO's whole + # computable set. cms2/cms130/cms137 have their second-engine evidence; cms165 runs on the + # corpus's stamped profiles (trustMetaProfile per measure, ADR-076 d1) and its WebChart + # ingest caveat governs the PHI phase, not the sandbox. reconcile-maui-mieweb.yml MUST match. + {key: "WORKWELL_OFFICIAL_MEASURES", value: "cms122,cms125,cms2,cms130,cms165,cms137"}, {key: "WORKWELL_MAUI_CORPUS_SIZE", value: "20000"}, {key: "WORKWELL_RUN_CHUNK_SIZE", value: "500"}, {key: "WORKWELL_SCHEDULER_ANCHOR_HOUR_UTC", value: "12"}, diff --git a/.github/workflows/reconcile-maui-mieweb.yml b/.github/workflows/reconcile-maui-mieweb.yml index 6ba403b9..fce61458 100644 --- a/.github/workflows/reconcile-maui-mieweb.yml +++ b/.github/workflows/reconcile-maui-mieweb.yml @@ -207,7 +207,7 @@ jobs: # and cms122/cms125 quietly revert to authored CQL with no signal anywhere. Caught by # Codex on #356 before it could happen; `official-flip-config.test.ts` now asserts the two # workflows agree, so this pair cannot drift again. - {key: "WORKWELL_OFFICIAL_MEASURES", value: "cms122,cms125"}, + {key: "WORKWELL_OFFICIAL_MEASURES", value: "cms122,cms125,cms2,cms130,cms165,cms137"}, {key: "WORKWELL_MAUI_CORPUS_SIZE", value: "20000"}, {key: "WORKWELL_RUN_CHUNK_SIZE", value: "500"}, {key: "WORKWELL_SCHEDULER_ANCHOR_HOUR_UTC", value: "12"}, diff --git a/CLAUDE.md b/CLAUDE.md index 1fe04174..41a24ece 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -119,13 +119,13 @@ plan, ADR-070 drives it, and the owner decisions are `LOCKED_DECISIONS.md` §4A retelling. Naming policy: repo documents say "Maui" and "the pilot group" only. - **MM-0 shipped** (#496–#500). **MM-1 is in progress:** U1 (#526, ADR-072 — the runnable rule, the calendar measurement period, the flip gate) and U2 (#528, ADR-073/074/075 — the 20,000-patient - corpus, multi-rate execution, outcome retention) merged 2026-09-05/06. Maui routes cms122 + cms125; - cms2, cms130, cms165 and cms137 are `official-pending` until MM-1c's second-engine sweep and a - `flip-gate` run clear each one. **No known-unverified measure is routed to the pilot**, the flip is a - reviewed workflow edit (ADR-045), and **cms165 must not be routed** until real blood pressures are - profile-stamped at ingest (ADR-076 d1 made profile trust per-measure, which is half the fix; - `MEASURES.md` and issue #533 carry the other half). U3 (#529) and the MM-1 open-flag work of - 2026-09-07 (ADR-076, issues #530–#537) are the newest entries — read `docs/JOURNAL.md`, not this line. + corpus, multi-rate execution, outcome retention) merged 2026-09-05/06. **Since 2026-09-08 (ADR-078, + owner decision) the Maui sandbox routes the ACO's whole computable set — cms122, cms125, cms2, + cms130, cms165 and cms137.** The flip is a reviewed workflow edit (ADR-045). Two caveats travel with + it and govern the PHI phase, not the sandbox: cms165 runs on the corpus's stamped profiles and needs + real blood pressures profile-stamped at ingest before real data (issue #533's ingest half); cms137 + stays routed unless the final rule removes Quality ID 305. An out-of-population official outcome no + longer opens a case (ADR-078 d2). Read `docs/JOURNAL.md` for the newest entries, not this line. - MM-2/3/4 are blocked on externals (ROADMAP §7). The milestones deliver a **sandbox**; the pilot's production/PHI phase is a separate `PRODUCTION_READINESS`-gated decision nothing in M-M authorizes. - M-C (packaging) is complete and published; M-E1 (occupational content) is deferred behind M-M, not diff --git a/backend-ts/src/case/case-logic.test.ts b/backend-ts/src/case/case-logic.test.ts index 17c4725b..79ef13a6 100644 --- a/backend-ts/src/case/case-logic.test.ts +++ b/backend-ts/src/case/case-logic.test.ts @@ -222,3 +222,26 @@ test("official-routed nextActionFor reads the official display table first", () else process.env.WORKWELL_OFFICIAL_MEASURES = prior; } }); + +test("planCaseUpsert: a subject OUTSIDE the initial population never opens a case, and closes an active one under its own reason (ADR-078)", () => { + const NOW = "2026-09-08T12:00:00.000Z"; + const out = { outOfPopulation: true }; + assert.deepEqual(planCaseUpsert(null, "MISSING_DATA", NOW, out), { op: "noop" }, "no case exists → nothing to open"); + assert.deepEqual( + planCaseUpsert({ status: "OPEN", currentOutcomeStatus: "MISSING_DATA", closedBy: null }, "MISSING_DATA", NOW, out), + { op: "update", disposition: "RESOLVED", status: "RESOLVED", closedAt: NOW, closedReason: "OUT_OF_POPULATION", closedBy: null }, + "an active case is closed by the system, with the reason that says why", + ); + assert.deepEqual( + planCaseUpsert({ status: "IN_PROGRESS", currentOutcomeStatus: "OVERDUE", closedBy: null }, "MISSING_DATA", NOW, out), + { op: "update", disposition: "RESOLVED", status: "RESOLVED", closedAt: NOW, closedReason: "OUT_OF_POPULATION", closedBy: null }, + "IN_PROGRESS too: an operator was working a patient the measure does not concern", + ); + assert.deepEqual( + planCaseUpsert({ status: "RESOLVED", currentOutcomeStatus: "COMPLIANT", closedBy: "someone" }, "MISSING_DATA", NOW, out), + { op: "noop" }, + "a closed case stays closed — no closed_at drift", + ); + // Without the flag the same status keeps opening cases: in-population MISSING_DATA is still a gap. + assert.equal(planCaseUpsert(null, "MISSING_DATA", NOW).op, "insert"); +}); diff --git a/backend-ts/src/case/case-logic.ts b/backend-ts/src/case/case-logic.ts index 478cc808..ddd726f7 100644 --- a/backend-ts/src/case/case-logic.ts +++ b/backend-ts/src/case/case-logic.ts @@ -229,7 +229,26 @@ export function planNextAction( } /** Decide how a case should be upserted from one outcome, given the existing row (or null). Pure. */ -export function planCaseUpsert(existing: ExistingCaseState | null, outcomeStatus: string, now: string): CaseUpsertPlan { +export function planCaseUpsert( + existing: ExistingCaseState | null, + outcomeStatus: string, + now: string, + opts: { outOfPopulation?: boolean } = {}, +): CaseUpsertPlan { + // A subject the official logic evaluated and found OUTSIDE the measure's initial population has + // nothing to chase: MISSING_DATA there is a result, not a gap (ADR-078). Never opens a case; a case + // that exists (the subject was in the population last period, or was carded before this rule) is + // closed by the system under its own reason, so the worklist stops carrying a non-diabetic on a + // diabetes measure. Until 2026-09-08 this fan-out opened one MEDIUM case per out-of-population + // subject per measure — ADR-043 recorded it as operational noise; six routed measures over 20,000 + // patients made it the worklist. + if (opts.outOfPopulation) { + if (!existing) return { op: "noop" }; + if (existing.status === "OPEN" || existing.status === "IN_PROGRESS") + return { op: "update", disposition: "RESOLVED", status: "RESOLVED", closedAt: now, closedReason: "OUT_OF_POPULATION", closedBy: null }; + return { op: "noop" }; + } + const disposition = dispositionFor(outcomeStatus); if (!existing) { diff --git a/backend-ts/src/cds/cards.test.ts b/backend-ts/src/cds/cards.test.ts index c4cfd565..21d35a04 100644 --- a/backend-ts/src/cds/cards.test.ts +++ b/backend-ts/src/cds/cards.test.ts @@ -313,3 +313,15 @@ test("an absence of data is a CARD, and it does not claim compliance", async () assert.doesNotMatch(card.detail!, /\bcompliant\b(?!\.)/i); assert.equal(card.suggestions, undefined); }); + +test("a subject the official logic found OUTSIDE the initial population gets no card (ADR-078)", async () => { + const outside = row("cms122", "MISSING_DATA", { + evidence: { expressionResults: [], official: { populationResults: { ipp: false, denom: false, denex: false, numer: false, denexcep: false } } }, + }); + assert.deepEqual(await buildComplianceCards([outside], opts()), [], "MISSING_DATA outside the population is a result, not a gap"); + // In the population and missing data is still a gap, and still carded. + const inside = row("cms122", "MISSING_DATA", { + evidence: { expressionResults: [], official: { populationResults: { ipp: true, denom: true, denex: false, numer: false, denexcep: false } } }, + }); + assert.equal((await buildComplianceCards([inside], opts())).length, 1); +}); diff --git a/backend-ts/src/cds/cards.ts b/backend-ts/src/cds/cards.ts index 286133a9..bf8f3f0a 100644 --- a/backend-ts/src/cds/cards.ts +++ b/backend-ts/src/cds/cards.ts @@ -20,6 +20,7 @@ import { deriveCell } from "../compliance/roster-vocabulary.ts"; import { deriveWhyFlagged } from "../case/case-detail-read-model.ts"; import { dispositionFor, nextActionFor, priorityFor } from "../case/case-logic.ts"; +import { officialMembership } from "../fhir/measure-report.ts"; import { MEASURE_CATALOG } from "../measure/measure-catalog.ts"; import { proposeOrders } from "../order/order-proposal.ts"; import { dedupeKeyFor, toServiceRequest, type ProposedOrder } from "../order/proposed-order.ts"; @@ -250,7 +251,12 @@ export async function buildComplianceCards( rows: readonly CardInput[], opts: CardOptions, ): Promise { - const open = rows.filter((r) => dispositionFor(r.status) === "OPEN" && !isDeprecated(r.measureId)); + // A subject the official logic found OUTSIDE the initial population gets no card (ADR-078): their + // MISSING_DATA is a result, and a card telling a clinician to "check eligibility" for a patient the + // measure does not concern is the noise ADR-067's card surface exists to avoid. + const open = rows.filter( + (r) => dispositionFor(r.status) === "OPEN" && !isDeprecated(r.measureId) && officialMembership(r.evidence)?.ipp !== false, + ); if (open.length === 0) return []; // ONE `proposeOrders` call for the whole subject, so its in-batch dedupe applies: two measures mapping diff --git a/backend-ts/src/run/run-pipeline.test.ts b/backend-ts/src/run/run-pipeline.test.ts index 46f12354..14b7e72f 100644 --- a/backend-ts/src/run/run-pipeline.test.ts +++ b/backend-ts/src/run/run-pipeline.test.ts @@ -1554,3 +1554,48 @@ test("measureDisplayName resolves an official-only measure that has no authored- // A genuinely unknown id degrades to itself rather than throwing. assert.equal(measureDisplayName("not-a-measure"), "not-a-measure"); }); + +test("ADR-078: a subject the official logic finds OUTSIDE the initial population opens no case, and an existing one is closed under OUT_OF_POPULATION", async () => { + const db = await createSqliteD1(join(tmpdir(), `workwell-pipeline-oop-${crypto.randomUUID()}.sqlite`)); + await db.exec(RUN_STORE_FLOOR_DDL.replace(/\n/g, " ")); + const caseStore = new SqliteCaseStore(db); + const captured: { eventType: string; payload?: unknown }[] = []; + // Two subjects on one measure. The first is a gap in the population; the second the logic evaluated + // and found outside it (a non-diabetic on a diabetes measure): same MISSING_DATA bucket, opposite meaning. + let outsideIds = new Set(); + const deps: RunPipelineDeps = { + runStore: new SqliteRunStore(db), + outcomeStore: new SqliteOutcomeStore(db), + caseStore, + engine: { + evaluate: async (input: { measureId: string; patientBundle: unknown }) => { + const id = (input.patientBundle as { entry: Array<{ resource: { id: string } }> }).entry[0]!.resource.id; + return outsideIds.has(id) + ? { outcome: "MISSING_DATA", evidence: { official: { populationResults: { ipp: false, denom: false, numer: false, denex: false, denexcep: false } } }, inInitialPopulation: false } + : { outcome: "OVERDUE", evidence: { official: { populationResults: { ipp: true, denom: true, numer: false, denex: false, denexcep: false } } }, inInitialPopulation: true }; + }, + logicVersionFor: () => "official-fqm:1.0.000:artifact:terminology", + } as unknown as RunPipelineDeps["engine"], + employees: EMPLOYEES.slice(0, 2), + actor: "cm@workwell.dev", + events: { async appendAudit(input) { captured.push({ eventType: input.eventType, payload: input.payload }); } }, + }; + const [first, second] = EMPLOYEES.slice(0, 2).map((e) => e.externalId) as [string, string]; + outsideIds = new Set([second]); + await executeManualRun(deps, { scopeType: "MEASURE", measureId: "audiogram", triggeredBy: "test" }); + const after1 = await caseStore.listCases({ limit: 100 }); + assert.deepEqual(after1.map((c) => c.employeeId).sort(), [first], "only the in-population gap opened a case"); + assert.equal(captured.filter((e) => e.eventType === "CASE_CREATED").length, 1); + + // Next run the first subject has left the population too: their case is closed by the system, audited. + outsideIds = new Set([first, second]); + captured.length = 0; + await executeManualRun(deps, { scopeType: "MEASURE", measureId: "audiogram", triggeredBy: "test" }); + const after2 = await caseStore.listCases({ limit: 100 }); + assert.equal(after2.length, 1, "no new case for either subject"); + assert.equal(after2[0]!.status, "RESOLVED"); + assert.equal(after2[0]!.closedReason, "OUT_OF_POPULATION"); + assert.equal(after2[0]!.closedBy, null, "a system closure, so a later in-population outcome may reopen it"); + assert.equal(captured.filter((e) => e.eventType === "CASE_RESOLVED").length, 1, "the closure is audited"); + assert.equal(captured.filter((e) => e.eventType === "CASE_CREATED").length, 0); +}); diff --git a/backend-ts/src/run/run-pipeline.ts b/backend-ts/src/run/run-pipeline.ts index 69892e63..f7a72a60 100644 --- a/backend-ts/src/run/run-pipeline.ts +++ b/backend-ts/src/run/run-pipeline.ts @@ -810,6 +810,8 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun plan: Awaited["plan"]>> | null; evaluatedNow: boolean; evaluationFailed: boolean; + /** The executor said this subject is OUTSIDE the initial population (ADR-078): a result, never a case. */ + outOfPopulation: boolean; } const pending: PendingOutcome[] = []; for (const item of chunkItems) { @@ -840,6 +842,7 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun : null; let evaluatedNow = true; // false ⇒ copied forward; true ⇒ a real (or attempted) CQL evaluation let evaluationFailed = false; + let outOfPopulation = false; // A failed batch outranks a cache hit. Unreachable today — ADR-040 §6 means an official-routed // measure is never reused, so a measure that could fail a batch never produces a `reuse` plan — but // the ordering is the difference between "wasteful" and "wrong" if that policy is lifted: a reused @@ -862,6 +865,10 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun (await deps.engine.evaluate({ measureId: item.measureId, patientBundle: bundle, evaluationDate: evalDate })); status = result.outcome; evidence = result.evidence; + // Read off the executor's own flag, not re-derived from evidence: the authored engine never + // sets it, so an authored MISSING_DATA keeps opening a case (it means "no record"), while an + // official `false` means the logic ran and the subject is not the measure's concern (ADR-078). + outOfPopulation = result.inInitialPopulation === false; // ADR-043 — record membership from the FINAL outcome, whichever path produced it (batch prefetch // or the individual fallback on this line). Reading it here rather than in the pre-pass is what // makes the roster complete before it is judged. A failed evaluation lands in `catch` below and @@ -884,7 +891,7 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun evaluationFailed = true; } } - pending.push({ item, period, status, evidence, plan, evaluatedNow, evaluationFailed }); + pending.push({ item, period, status, evidence, plan, evaluatedNow, evaluationFailed, outOfPopulation }); } // Invariant 7 / memory: this chunk's bundles are released here, before anything else happens and // certainly before the next chunk is built. @@ -927,7 +934,7 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun planned.progress.failures = failures; for (const [index, entry] of pending.entries()) { - const { item, period, status, evidence, plan, evaluatedNow, evaluationFailed } = entry; + const { item, period, status, evidence, plan, evaluatedNow, evaluationFailed, outOfPopulation } = entry; const recorded = records[index]!; // #263: cache the fingerprint of a SUCCESSFUL real evaluation so a future run can reuse it. Never // cache an engine-failure MISSING_DATA (we must not copy an error forward), and never re-cache a @@ -950,9 +957,17 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun // (subject, measure, period) key (a fresh waiver excuses an existing open case), because EXCLUDED // with NO existing case would INSERT a new EXCLUDED case and re-pollute the gate. The outcome above // is ALWAYS persisted (CQL is the sole compliance authority — ADR-008). Empty/absent segments ⇒ all. + const caseKey = `${item.employee.externalId}|${item.measureId}|${period}`; + // (3) OUT OF THE INITIAL POPULATION (ADR-078) — the same close-only shape as EXCLUDED: a subject the + // official logic found outside the measure's population never opens a case, and an active one (they + // were in the population last period, or were carded before this rule) is closed by the system + // under `OUT_OF_POPULATION`. Until 2026-09-08 this was ADR-043's recorded fan-out: every + // non-diabetic opened a MEDIUM CMS122 case, and six routed measures made that the worklist. const closeOnly = status === "COMPLIANT" || - (status === "EXCLUDED" && activeCaseKeys.has(`${item.employee.externalId}|${item.measureId}|${period}`)); + (status === "EXCLUDED" && activeCaseKeys.has(caseKey)) || + (outOfPopulation && activeCaseKeys.has(caseKey)); + const outOfPopulationWithNoCase = outOfPopulation && !activeCaseKeys.has(caseKey); // Live WebChart subjects are display-applicable (their roster cells show real chips) but must NOT // OPEN cases: rerun-to-verify returns a non-mutating 409 for `wc|` subjects until fetch-one-patient // lands, so a newly-created wc case would be un-closeable. Case CREATION eligibility is therefore @@ -982,13 +997,13 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun (applicableMemo ??= isApplicable(item.employee, item.measureId, deps.segments ?? [])); // `wc|` subjects are excluded deliberately, not by oversight: they never open cases at all (the // rerun-to-verify 409 above), so counting them here would report a gap no segment edit can close. - if (!closeOnly && !isLiveWebChartSubject && NON_COMPLIANT.has(status) && !segmentApplicable()) { + if (!closeOnly && !outOfPopulation && !isLiveWebChartSubject && NON_COMPLIANT.has(status) && !segmentApplicable()) { gatedBySegment.evaluations++; gatedBySegment.subjects.add(item.employee.externalId); if (gatedBySegment.sites.size < 12) gatedBySegment.sites.add(item.employee.site ?? "(no site)"); gatedBySegment.measures.add(item.measureId); } - if (deps.caseStore && (closeOnly || (!isLiveWebChartSubject && segmentApplicable()))) { + if (deps.caseStore && !outOfPopulationWithNoCase && (closeOnly || (!isLiveWebChartSubject && segmentApplicable()))) { const upserted = await deps.caseStore.upsertFromOutcome({ runId: runId, subjectId: item.employee.externalId, @@ -996,6 +1011,7 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun evaluationPeriod: period, outcomeStatus: status, evidence, + outOfPopulation, }); // Audit the case transition (Fable H1 — the population pipeline previously wrote NO case audit // events, violating the "every state change writes audit_event" hard rule). Idempotent diff --git a/backend-ts/src/stores/case-store.ts b/backend-ts/src/stores/case-store.ts index 1cefee92..c5280c60 100644 --- a/backend-ts/src/stores/case-store.ts +++ b/backend-ts/src/stores/case-store.ts @@ -34,6 +34,12 @@ export interface UpsertCaseInput { * multi-rate measure (ADR-074: the rate the subject missed); never persisted by the case store. */ evidence?: unknown; + /** + * The official logic evaluated this subject and found them OUTSIDE the initial population (ADR-078). + * Never creates a case; resolves an active one with `closed_reason='OUT_OF_POPULATION'` (a system + * closure). The run pipeline sets it from the persisted official evidence; the status stays MISSING_DATA. + */ + outOfPopulation?: boolean; } /** diff --git a/backend-ts/src/stores/postgres/case-store-postgres.ts b/backend-ts/src/stores/postgres/case-store-postgres.ts index 5c910869..10cc7d3a 100644 --- a/backend-ts/src/stores/postgres/case-store-postgres.ts +++ b/backend-ts/src/stores/postgres/case-store-postgres.ts @@ -73,7 +73,9 @@ export class PgCaseStore implements CaseStore { const priority = priorityFor(input.outcomeStatus); const computedAction = nextActionFor(input.outcomeStatus, input.measureId, input.evidence); const planFrom = (row: CaseRow | null) => - planCaseUpsert(row ? { status: row.status, currentOutcomeStatus: row.current_outcome_status, closedBy: row.closed_by } : null, input.outcomeStatus, now); + planCaseUpsert(row ? { status: row.status, currentOutcomeStatus: row.current_outcome_status, closedBy: row.closed_by } : null, input.outcomeStatus, now, { + outOfPopulation: input.outOfPopulation, + }); // An operator's instruction outlives a run that learned nothing new (`planNextAction`). const actionFrom = (row: CaseRow | null) => planNextAction( diff --git a/backend-ts/src/stores/sqlite/case-store-sqlite.ts b/backend-ts/src/stores/sqlite/case-store-sqlite.ts index 6b7923f2..6761e29d 100644 --- a/backend-ts/src/stores/sqlite/case-store-sqlite.ts +++ b/backend-ts/src/stores/sqlite/case-store-sqlite.ts @@ -88,6 +88,7 @@ export class SqliteCaseStore implements CaseStore { existing ? { status: existing.status, currentOutcomeStatus: existing.current_outcome_status, closedBy: existing.closed_by } : null, input.outcomeStatus, now, + { outOfPopulation: input.outOfPopulation }, ); if (plan.op === "noop") return null; @@ -126,6 +127,7 @@ export class SqliteCaseStore implements CaseStore { existing ? { status: existing.status, currentOutcomeStatus: existing.current_outcome_status, closedBy: existing.closed_by } : null, input.outcomeStatus, now, + { outOfPopulation: input.outOfPopulation }, ); // The winner of the race may have written an operator-owned action between our read and theirs. action = planNextAction( @@ -195,6 +197,7 @@ export class SqliteCaseStore implements CaseStore { { status: existing.status, currentOutcomeStatus: existing.current_outcome_status, closedBy: existing.closed_by }, input.outcomeStatus, now, + { outOfPopulation: input.outOfPopulation }, ); if (plan.op !== "update") return null; action = planNextAction( diff --git a/docs/ADR_INDEX.md b/docs/ADR_INDEX.md index bc948e3b..3688b18f 100644 --- a/docs/ADR_INDEX.md +++ b/docs/ADR_INDEX.md @@ -6,14 +6,15 @@ > `grep -o '^#\+ ADR-[0-9]*.*' docs/DECISIONS.md`, newest first. If the highest number here is lower > than the highest there, this file is stale. > -> **`·archived`** (14 of 76) means the BODY moved to `docs/archive/DECISIONS_ARCHIVE.md` — superseded, +> **`·archived`** (14 of 77) means the BODY moved to `docs/archive/DECISIONS_ARCHIVE.md` — superseded, > or a historical *finding* rather than a decision that governs. `DECISIONS.md` keeps every heading plus -> a pointer, so every cross-reference still resolves. The 62 unmarked titles are the record that governs. +> a pointer, so every cross-reference still resolves. The 63 unmarked titles are the record that governs. > > **ADR-033 does not exist** — the sequence runs 031, 032, 034. Verified absent 2026-07-29. Do not reuse it. ## Titles (newest first) +- ADR-078: the sandbox routes the ACO's whole computable set — and a subject outside a measure's population is a result, not a case - ADR-077: a report is refused rather than rendered from rows that may be incomplete — and a dashboard rate is the evidence's rate, shown apart from the workflow's - ADR-076: profile trust is a per-measure fact, and an operator's next action outranks the wording table - ADR-075: the pilot's roster is a generated corpus the deployment composes lazily, and evaluation runs in subject chunks diff --git a/docs/DATA_MODEL_CONTRACTS.md b/docs/DATA_MODEL_CONTRACTS.md index 892e2e88..f40d5f4c 100644 --- a/docs/DATA_MODEL_CONTRACTS.md +++ b/docs/DATA_MODEL_CONTRACTS.md @@ -35,6 +35,12 @@ SQLite floor and the Pg ceiling read the current row and apply the shared pure ` `IN_PROGRESS`) counts both — otherwise a reconfirmed IN_PROGRESS case would silently drop out of the hierarchy/programs open-case count. - **No `closed_at` drift.** A COMPLIANT outcome on an already-terminal case is a no-op. +- **A subject OUTSIDE the initial population never opens a case (ADR-078).** The run pipeline sets + `outOfPopulation` on the upsert from the executor's own `inInitialPopulation: false`; the outcome is + still persisted as MISSING_DATA (CQL is authoritative), but `planCaseUpsert` returns a no-op where + no case exists and closes an active one with `status=RESOLVED`, `closed_reason='OUT_OF_POPULATION'`, + `closed_by=NULL` (a system closure, audited `CASE_RESOLVED`). In-population MISSING_DATA still opens + a case. Before this, every non-diabetic opened a CMS122 case (ADR-043's recorded fan-out). - The upsert returns an `UpsertedCase` (a `CaseRecord` superset carrying a `disposition` of `CREATED | UPDATED | REOPENED | RESOLVED | EXCLUDED | UNCHANGED`). The run pipeline emits a matching `CASE_*` audit event for every disposition except `UNCHANGED` (an idempotent re-confirm of the same open diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index 31e0a91d..a9f7c247 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -18,6 +18,60 @@ > > **Sequence note:** ADR-033 does not exist — verified absent, and the number must not be reused. +## ADR-078: the sandbox routes the ACO's whole computable set — and a subject outside a measure's population is a result, not a case + +**Date:** 2026-09-08. **Status:** accepted — an OWNER decision, recorded in `LOCKED_DECISIONS.md` §4A.2 +as a SINCE note. Amends ADR-043 (the case fan-out it recorded) and reads ADR-072's cms165 consequence +and §4A.5's "known-unverified" bar as governing the PHI phase. + +### Context + +Eight days before the pilot group's quality lead was told the sandbox held her ACO's measure set, it +showed two. cms2, cms130, cms165 and cms137 were vendored, MADiE-gated (36/36, 64/64, 68/68, 45/45), +runnable under ADR-072, and unrouted, each waiting on a sequenced precondition: cms2's seven +cross-engine disagreements (diagnosed 2026-09-07, #538), cms130's sweep (63/64, #539), cms137's flip +after those two and after the final rule on Quality ID 305, and cms165's WebChart ingest half (#533). +The owner's priority is every measure the pilot group sent us working and visible in the sandbox, and +the sandbox evaluates a generated corpus, not WebChart data. + +Routing all six exposed the second problem at scale. An official outcome outside the initial +population persists as MISSING_DATA and, through `dispositionFor`, opened a MEDIUM "collect the +documentation" case — ADR-043 recorded this as "real operational noise" and left it. With two measures +it was 16,581 open cases, 91–97 % of them patients the measure does not concern; with six it would have +been the whole worklist. + +### Decision + +1. **The Maui sandbox routes cms122, cms125, cms2, cms130, cms165 and cms137** (`deploy-maui-mieweb.yml` + and `reconcile-maui-mieweb.yml`, which must agree). Evidence attached to the flip: cms2's whole-roster + gate (36/36, 17,795/20,000 in the initial population, 5,413 actionable, 0 errors), cms137's + (`docs/evidence/FLIP_GATE_2026-09-07_CMS137.md`), the cross-engine sweeps, and the credentialed + gates for cms130 and cms165 run through the new `flip-gate.yml` workflow, because their pinned + sidecars are VSAC-completed and do not resolve locally. +2. **A subject the executor finds outside the initial population never opens a case.** The pipeline + reads the executor's own `inInitialPopulation: false` — never re-derived from evidence, so an + authored MISSING_DATA ("no record") keeps opening cases — and treats it as a close-only outcome: + no case where none exists; an active case is closed by the system with + `closed_reason='OUT_OF_POPULATION'`, audited `CASE_RESOLVED`, reopenable by a later in-population + outcome. The CDS card surface applies the same rule off the persisted evidence. The outcome row is + unchanged: CQL stays the authority (ADR-008), and the roster shows OUT_OF_POPULATION (ADR-077 d7). +3. **The two conditions the locked decision named move to the PHI phase.** cms137 stays routed on + the sandbox unless the final rule removes 305, at which point it is un-routed by the same workflow + edit. cms165 runs on the corpus's stamped profiles (ADR-076 d1, #539); before real data it needs + every QI-Core profile it retrieves stamped at ingest and WebChart's BP status to arrive final + (#533). Neither is a sandbox blocker, and both are written into the PHI readiness gate. + +### Consequences + +- The nightly run evaluates 120,000 subject-measure pairs instead of 40,000. The first such run is the + measurement; if it does not fit the anchor window the chunk size and the anchor hour are the knobs. +- Existing out-of-population cases on the sandbox — roughly 15,000 — are closed by the first run after + deploy under `OUT_OF_POPULATION`, each with an audit event. That is one large ledger write, once. +- cms165's cross-engine number is still open (#532). Routing it on the sandbox does not settle that + question and does not claim to; the measure's own MADiE deck is the verification it carries. +- The flip gate is now runnable where its sidecars resolve (`flip-gate.yml`), which removes the + "run it somewhere credentialed" instruction from the runbook's list of things a person has to know. + ## ADR-077: a report is refused rather than rendered from rows that may be incomplete — and a dashboard rate is the evidence's rate, shown apart from the workflow's **Status:** Accepted (2026-09-08). Amends ADR-031, ADR-073 and ADR-074 d12. diff --git a/docs/DEPLOY.md b/docs/DEPLOY.md index 64702110..9881b507 100644 --- a/docs/DEPLOY.md +++ b/docs/DEPLOY.md @@ -311,10 +311,15 @@ Per measure, per stack: > roster's: at 200 subjects CMS137 has 8 in its denominator and 0 engaged, which trips the "a rate nobody > reaches" finding; at 2,000 it is 60 and 8, and the gate passes. > -> **cms165 has a blocker `flip-gate` does not detect.** The executor ignores `meta.profile`, and -> cms165 identifies its BP reading by profile alone — so on any patient with other Observations the -> numerator silently reads false. The synthetic roster emits one Observation per subject and therefore -> cannot show it. Do not route cms165 on the strength of a green gate alone (ADR-072, consequences). +> **cms165 has a blocker `flip-gate` does not detect on REAL data.** cms165 identifies its BP reading +> by profile alone; since ADR-076 d1 it runs with `trustMetaProfile` and the corpus stamps every profile +> its retrieves name, and since #539 `prepareForQiCore` stamps `us-core-blood-pressure` from a +> resource's own codes. That is what lets the sandbox route it (ADR-078). It is NOT sufficient for +> WebChart data: the other QI-Core profiles must arrive stamped at ingest, and WebChart exports BP panels +> with `status: "unknown"` (issue #533's open half). A green gate over the corpus says nothing about that. +> +> **The credentialed gate runs in CI:** `.github/workflows/flip-gate.yml` (manual) vendors the measure's +> sidecar with the VSAC secret, sweeps the Maui roster, and uploads `.flip-gate/-.json`. > > **A reading that did not run is never a pass.** Every pilot measure's MADiE deck is vendored into the > tree (2026-09-05), but the deck and the roster both need the measure's terminology sidecar, which is diff --git a/docs/JOURNAL.md b/docs/JOURNAL.md index 5b721421..b797e257 100644 --- a/docs/JOURNAL.md +++ b/docs/JOURNAL.md @@ -1,5 +1,38 @@ # Journal +## 2026-09-08 (later) — all six measures on the sandbox, and a patient outside a measure is not a case (ADR-078) + +The owner looked at the programs page with two measure cards on it and asked why, when the pilot +group sent us six. The honest answer was sequencing: cms2, cms130, cms165 and cms137 were vendored, +gated against MADiE, runnable since ADR-072, and each parked behind a precondition I had set — cms2's +disagreements (diagnosed yesterday), cms130's sweep (done yesterday), cms137's order in the queue and +the final rule on 305, and cms165's WebChart ingest half. The owner's decision is that a sandbox on a +generated corpus should show the pilot group its whole set now, and that the conditions I was holding +the flips on are conditions on real data. Recorded as ADR-078 and a SINCE note in the locked decisions. + +**The flip is the workflow edit ADR-045 says it is**, in both Maui workflows, plus CMS137 added to the +deploy's vendor step (it was the one pilot measure the build did not vendor). cms2's gate over all +20,000 corpus patients reads 36/36 MADiE, 17,795 in the initial population, 5,413 actionable, zero +errors (`docs/evidence/FLIP_GATE_2026-09-08_CMS2.md`); cms137's was banked yesterday. cms130 and cms165 +pin VSAC-completed sidecars that do not resolve on a laptop, so the gate for them now runs where the +credential is: a manual `flip-gate.yml` workflow that vendors with the secret, sweeps the roster and +uploads the report. That removes a "run it somewhere credentialed" instruction that only existed in +prose. + +**Routing six measures made ADR-043's noise the worklist.** An official outcome outside the initial +population persists as MISSING_DATA and opened a MEDIUM case, so 91–97 % of the sandbox's 16,581 open +cases were patients the measure does not concern — every non-diabetic on CMS122, every man on CMS125. +The pipeline now reads the executor's own `inInitialPopulation: false` and treats it as close-only: +no case where none exists, and an active one closed by the system under `OUT_OF_POPULATION`, audited, +reopenable. The CDS card surface applies the same rule. The authored engine never sets the flag, so an +authored MISSING_DATA still means "no record" and still opens a case. The first run after deploy will +close roughly fifteen thousand cases in one pass; that is the ledger doing its job once. + +**What the flip does not settle.** cms165's cross-engine number is still an open question (#532), and +its ingest half (#533) now gates the PHI phase rather than the sandbox. cms137 comes off again by the +same edit if the final rule removes 305. The nightly run is now 120,000 pairs; the first one is the +measurement. + ## 2026-09-08 — the numbers a quality lead can see are true or refused (ADR-077) An external review of the Maui sandbox, taken against `564d5d93`, found four ways a number could be diff --git a/docs/LOCKED_DECISIONS.md b/docs/LOCKED_DECISIONS.md index f842496a..7ae537fe 100644 --- a/docs/LOCKED_DECISIONS.md +++ b/docs/LOCKED_DECISIONS.md @@ -78,6 +78,14 @@ document's §4 verification set stays the bar per §4 decision 2 above). > is still outstanding is the MM-1c second-engine sweep and a `flip-gate` run in a context that resolves > the terminology sidecar. Decision 5 below (no known-unverified measure routed to the pilot) is > unchanged and is what keeps them unrouted until that sweep is done. + + > **SINCE (2026-09-08, ADR-078 — owner decision):** the sandbox routes ALL six — cms122, cms125, + > cms2, cms130, cms165, cms137. The owner's priority is every measure the pilot group sent us working + > and visible in the sandbox; cms2's disagreements are diagnosed, cms130 is swept, cms137 is gated + > over the whole roster, and cms165 runs on the corpus's stamped profiles. The two conditions this + > decision named — 305 surviving the final rule for cms137, and profile-stamped real blood pressures + > for cms165 — now gate the PHI phase rather than the sandbox flip. Decision 5's "known-unverified" + > bar is read the same way: it applies to what the pilot runs over real data. 3. **Cards resolve, not alert** — order suggestions (gated on APPROVED terminology mappings; **an order is a proposal and never changes compliance** — the gap closes only when the qualifying result data arrives and CQL re-evaluates) and exception documentation (structured data the measure logic reads diff --git a/docs/MEASURES.md b/docs/MEASURES.md index a1168d8a..1ba68e2e 100644 --- a/docs/MEASURES.md +++ b/docs/MEASURES.md @@ -21,12 +21,13 @@ Runnable (full CQL): **14** — 4 OSHA + 5 HEDIS + 3 immunization panel + 2 CMS (`cms2`, `cms130`, `cms165`, and since 2026-09-06 `cms137` — the Draft `cms137v14` placeholder was retired into it by the ADR-071 seed, the way `cms2v15` was) have no authored CQL and no synthetic binding: they are runnable only where the deployment names them in `WORKWELL_OFFICIAL_MEASURES`, and `classifyRunnable` -reports them as `official-pending` everywhere else. Today that is **nowhere** — the Maui profile lists all -six pilot measures, but only `cms122` and `cms125` are routed, so CMS2, CMS130, CMS165 and CMS137 are -Active, gated and still not executing. All four pass their full MADiE deck against the runtime (36/36, -64/64, 68/68 and 45/45, zero errors), so what remains is the MM-1c second-engine sweep and a `flip-gate` -run where the terminology sidecar resolves — without it every value set expands empty and the gate reports -a zero initial population (locked decision §4A.5). **CMS137 is multi-rate** (Initiation and Engagement, +reports them as `official-pending` everywhere else. **Since 2026-09-08 the Maui sandbox routes all +six** (ADR-078, owner decision): cms122, cms125, cms2, cms130, cms165 and cms137, so CMS2, CMS130, +CMS165 and CMS137 are Active, gated AND executing there; TWH still routes cms122 and cms125 only. All +four pass their full MADiE deck against the runtime (36/36, 64/64, 68/68 and 45/45, zero errors); cms2's +whole-roster gate reads 17,795 of 20,000 in the initial population and 5,413 actionable +(`docs/evidence/FLIP_GATE_2026-09-08_CMS2.md`), and the credentialed gates for cms130 and cms165 run +through `.github/workflows/flip-gate.yml`. **CMS137 is multi-rate** (Initiation and Engagement, ADR-074): one catalog row, one outcome per patient reduced to the worst rate, and every rate persisted and exported — and the roster cell, the case detail, the case's next action and the CDS card name the rate the patient actually missed ("without treatment initiation within 14 days" or "initiated but not engaged @@ -41,8 +42,8 @@ banked (2026-09-07):** `flip-gate --measure cms137 --subjects all` over all 20,0 both rates alive (numerators 231 and 81) and the effectivePeriod covering the measured year — evidence FOR the flip on the full roster, not a 2,000-subject sample (`docs/evidence/FLIP_GATE_2026-09-07_CMS137.md`). -Its flip remains the owner's workflow edit, sequenced after cms2 and cms130 (ADR-072 D1), and the 2027 -pilot year still needs the MM-1d re-vendor. +Its flip landed with the others on 2026-09-08 (ADR-078); it is un-routed by the same workflow edit if the +final rule removes Quality ID 305, and the 2027 pilot year still needs the MM-1d re-vendor. **CMS2's verification debt is paid (2026-09-07).** Its seven cross-engine disagreements, open and unexplained since 2026-08-04, are proven to one cause: the Java engine takes a medication order's start @@ -87,7 +88,10 @@ is **362 of 387 across eight measures**, CMS165 deliberately excluded. > And CMS165's own cross-engine sweep is unexplained (#532). The failure mode is louder now, though not unconditionally: the batch-level retrieve refusal fires > only for a roster of more than one subject, so a nightly run would refuse while `/simulate` and > rerun-to-verify would quietly return MISSING_DATA. Better than a plausible wrong number, and not a -> guarantee. **Do not route cms165.** +> guarantee. **cms165 is routed on the SANDBOX since 2026-09-08 (ADR-078)** — it runs there because the +> corpus stamps every profile it retrieves and #539 stamps a blood pressure from its own codes — and +> **must not be routed over WebChart data** until the ingest half of #533 (every QI-Core profile stamped +> at ingest, BP status arriving final) is closed. That condition now sits in the PHI readiness gate. **An officially routed measure is scored over the calendar year** containing the evaluation date, not a rolling 365-day window (ADR-072). The vendored artifacts are a **2026 vintage** and the pilot year is 2027, diff --git a/docs/evidence/FLIP_GATE_2026-09-08_CMS2.md b/docs/evidence/FLIP_GATE_2026-09-08_CMS2.md new file mode 100644 index 00000000..6fedc142 --- /dev/null +++ b/docs/evidence/FLIP_GATE_2026-09-08_CMS2.md @@ -0,0 +1,31 @@ +# CMS2's flip gate over the whole Maui roster + +Date: 2026-09-08. Command, from `backend-ts`: + +```bash +WORKWELL_INSTANCE=maui WORKWELL_MAUI_CORPUS_SIZE=20000 WORKWELL_OFFICIAL_MEASURES=cms122,cms125 \ + pnpm flip-gate --measure cms2 --evaluation-date 2026-12-31 --subjects all +``` + +Report: `backend-ts/.flip-gate/cms2-2026-12-31.json` (gitignored; the numbers below are its content). + +| reading | result | +|---|---| +| MADiE — the steward's own expected vectors | **36 of 36** agree, 0 disagree | +| Roster — the artifact over the deployment's own 20,000 corpus patients, routed as `cms122,cms125,cms2` | **17,795 in the initial population and denominator**, 5,413 actionable, **0 evaluation errors** | +| Outcome distribution | COMPLIANT 11,884 · OVERDUE 5,413 · EXCLUDED 498 · MISSING_DATA 2,205 | +| `effectivePeriod` | covers the measured year (2026) | + +Verdict printed by the gate: *the three readings agree — evidence FOR the flip; the flip itself is a +workflow edit a human still makes.* The 2,205 MISSING_DATA are subjects outside the initial population +(no qualifying encounter or outside the age band), which the roster shows as OUT_OF_POPULATION and +which open no case (ADR-077 d7, ADR-078 d2). + +The run emitted ~18,700 `Failed to locate element for ServiceRequest.performed` lines on stderr — the +cql-exec-fhir model walker warning once per subject that CMS2's ELM reads a path QI-Core's +`ServiceRequest` does not carry. It is noise, not an error: the same warning appears on the MADiE deck, +which agrees 36/36. + +Second-engine evidence for the same measure: `CROSS_ENGINE_2026-09-07_CMS2.md` (29 of 36 agree; the +seven disagreements diagnosed to one medication-period helper, on which MADiE's own expected results +side with our engine). From 0cb3ebdc36acef49bafee15d974e5100936eaf2e Mon Sep 17 00:00:00 2001 From: Taleef Date: Tue, 8 Sep 2026 13:29:58 -0400 Subject: [PATCH 2/6] ci(deploy): no apostrophe inside the jq program that builds the Maui env array The flip comment sat inside a single-quoted jq string and its apostrophe ended the string; the deploy-helper shell test caught it before merge, which is what it exists for. --- .github/workflows/deploy-maui-mieweb.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/deploy-maui-mieweb.yml b/.github/workflows/deploy-maui-mieweb.yml index 309b285e..d3aeae38 100644 --- a/.github/workflows/deploy-maui-mieweb.yml +++ b/.github/workflows/deploy-maui-mieweb.yml @@ -277,10 +277,11 @@ jobs: {key: "WORKWELL_INSTANCE", value: "maui"}, {key: "WORKWELL_SCHEDULER_ENABLED", value: "true"}, {key: "WORKWELL_VSAC_API_KEY", value: $vsac_api_key}, - # THE FLIP (ADR-045; ADR-078, owner decision 2026-09-08): the sandbox routes the ACO's whole + # THE FLIP (ADR-045; ADR-078, owner decision 2026-09-08): the sandbox routes the whole ACO # computable set. cms2/cms130/cms137 have their second-engine evidence; cms165 runs on the - # corpus's stamped profiles (trustMetaProfile per measure, ADR-076 d1) and its WebChart + # profiles the corpus stamps (trustMetaProfile per measure, ADR-076 d1) and its WebChart # ingest caveat governs the PHI phase, not the sandbox. reconcile-maui-mieweb.yml MUST match. + # No apostrophes in this block: it sits inside a single-quoted jq program. {key: "WORKWELL_OFFICIAL_MEASURES", value: "cms122,cms125,cms2,cms130,cms165,cms137"}, {key: "WORKWELL_MAUI_CORPUS_SIZE", value: "20000"}, {key: "WORKWELL_RUN_CHUNK_SIZE", value: "500"}, From 0f77d568f1f8b98f11ba10363a02cecfe67175ab Mon Sep 17 00:00:00 2001 From: Taleef Date: Tue, 8 Sep 2026 13:32:56 -0400 Subject: [PATCH 3/6] fix(run): the out-of-population closure says why in its audit event, the card surface reads every rate, and the store contract pins the flag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Own review of the flip branch. CASE_RESOLVED now carries closedReason, so ~15,000 OUT_OF_POPULATION closures are not read as auto-resolves in the ledger; the CDS card filter reads every rate's initial population, the same reading the pipeline's inInitialPopulation takes on a multi-rate measure; the case-store contract pins open→closed-once→reopen on both floors; the gate workflow sets pipefail so tee cannot mask the CLI exit code; and two stale comments name the new closure path. --- .github/workflows/flip-gate.yml | 4 ++++ backend-ts/src/cds/cards.test.ts | 11 ++++++++++ backend-ts/src/cds/cards.ts | 12 ++++++----- backend-ts/src/run/run-pipeline.ts | 11 +++++++++- backend-ts/src/stores/store-contract.ts | 28 +++++++++++++++++++++++++ 5 files changed, 60 insertions(+), 6 deletions(-) diff --git a/.github/workflows/flip-gate.yml b/.github/workflows/flip-gate.yml index 4505c865..b8cb3ef4 100644 --- a/.github/workflows/flip-gate.yml +++ b/.github/workflows/flip-gate.yml @@ -95,12 +95,16 @@ jobs: run: git diff --exit-code backend-ts/measures/official - name: Gate + shell: bash env: WORKWELL_INSTANCE: maui WORKWELL_MAUI_CORPUS_SIZE: "20000" WORKWELL_RUN_CHUNK_SIZE: "500" WORKWELL_OFFICIAL_MEASURES: ${{ inputs.routed }} run: | + # pipefail, or `tee` would mask the CLI's own exit code (2 on a usage error) and the failure + # would surface later as "ls: cannot access .flip-gate" with the real message buried in stderr. + set -o pipefail pnpm flip-gate --measure "${{ inputs.measure }}" --evaluation-date "${{ inputs.evaluation_date }}" --subjects "${{ inputs.subjects }}" 2> gate-stderr.log | tee gate-report.txt echo "::group::stderr (engine warnings, deduplicated)" sort gate-stderr.log | uniq -c | sort -rn | head -40 diff --git a/backend-ts/src/cds/cards.test.ts b/backend-ts/src/cds/cards.test.ts index 21d35a04..8f700b9f 100644 --- a/backend-ts/src/cds/cards.test.ts +++ b/backend-ts/src/cds/cards.test.ts @@ -325,3 +325,14 @@ test("a subject the official logic found OUTSIDE the initial population gets no }); assert.equal((await buildComplianceCards([inside], opts())).length, 1); }); + +test("multi-rate: a subject in ANY rate's initial population is still carded — only outside every rate is a result (ADR-078)", async () => { + const rateIn = { ipp: true, denom: true, denex: false, numer: false, denexcep: false }; + const rateOut = { ipp: false, denom: false, denex: false, numer: false, denexcep: false }; + // Rate 1 says outside, rate 2 says inside: the pipeline's `inInitialPopulation` is true for this + // subject, so the card surface must agree and card them. + const partly = row("cms137", "MISSING_DATA", { evidence: { expressionResults: [], official: { populationResults: rateOut, rates: [rateOut, rateIn] } } }); + assert.equal((await buildComplianceCards([partly], opts())).length, 1, "in rate 2's population → carded"); + const wholly = row("cms137", "MISSING_DATA", { evidence: { expressionResults: [], official: { populationResults: rateOut, rates: [rateOut, rateOut] } } }); + assert.deepEqual(await buildComplianceCards([wholly], opts()), [], "outside every rate → no card"); +}); diff --git a/backend-ts/src/cds/cards.ts b/backend-ts/src/cds/cards.ts index bf8f3f0a..a94fe595 100644 --- a/backend-ts/src/cds/cards.ts +++ b/backend-ts/src/cds/cards.ts @@ -20,7 +20,7 @@ import { deriveCell } from "../compliance/roster-vocabulary.ts"; import { deriveWhyFlagged } from "../case/case-detail-read-model.ts"; import { dispositionFor, nextActionFor, priorityFor } from "../case/case-logic.ts"; -import { officialMembership } from "../fhir/measure-report.ts"; +import { membershipRatesFor, officialMembership } from "../fhir/measure-report.ts"; import { MEASURE_CATALOG } from "../measure/measure-catalog.ts"; import { proposeOrders } from "../order/order-proposal.ts"; import { dedupeKeyFor, toServiceRequest, type ProposedOrder } from "../order/proposed-order.ts"; @@ -253,10 +253,12 @@ export async function buildComplianceCards( ): Promise { // A subject the official logic found OUTSIDE the initial population gets no card (ADR-078): their // MISSING_DATA is a result, and a card telling a clinician to "check eligibility" for a patient the - // measure does not concern is the noise ADR-067's card surface exists to avoid. - const open = rows.filter( - (r) => dispositionFor(r.status) === "OPEN" && !isDeprecated(r.measureId) && officialMembership(r.evidence)?.ipp !== false, - ); + // measure does not concern is the noise ADR-067's card surface exists to avoid. "Outside" is EVERY + // rate's initial population, the same reading the pipeline's `inInitialPopulation` takes on a + // multi-rate measure (own review): official evidence present, and no rate admits the subject. + const outsidePopulation = (r: CardInput): boolean => + officialMembership(r.evidence) !== null && membershipRatesFor(r, r.measureId).every((m) => !m.ipp); + const open = rows.filter((r) => dispositionFor(r.status) === "OPEN" && !isDeprecated(r.measureId) && !outsidePopulation(r)); if (open.length === 0) return []; // ONE `proposeOrders` call for the whole subject, so its in-batch dedupe applies: two measures mapping diff --git a/backend-ts/src/run/run-pipeline.ts b/backend-ts/src/run/run-pipeline.ts index f7a72a60..213d5610 100644 --- a/backend-ts/src/run/run-pipeline.ts +++ b/backend-ts/src/run/run-pipeline.ts @@ -626,7 +626,8 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun activeCaseKeys.add(`${c.employeeId}|${c.measureId}|${c.evaluationPeriod}`); } } catch { - /* a read failure only means EXCLUDED stays applicability-gated — never abort the run */ + /* a read failure only means EXCLUDED stays applicability-gated and an out-of-population + closure (ADR-078) waits for the next run — never abort the run */ } } } @@ -854,6 +855,10 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun evidence = plan.evidence; evaluatedNow = false; skipped++; + // `outOfPopulation` stays false on a reuse: the cache carries no executor flag. Unreachable + // today (ADR-040 §6 keeps every official-routed measure out of the cache), and if that policy is + // lifted the flag must be derived from the cached `evidence.official`, or a reused + // out-of-population MISSING_DATA would open the case ADR-078 removed. } else { try { // A measure whose whole roster was evaluated in one official batch above is read from there. A @@ -1042,6 +1047,10 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun disposition: upserted.disposition, outcomeStatus: status, status: upserted.status, + // WHY a closure closed — AUTO_RESOLVED, EXCLUDED or OUT_OF_POPULATION (ADR-078). Without + // it ~15,000 out-of-population closures read like auto-resolves in the ledger, and an + // auditor would have to join `cases` to tell them apart (own review). + ...(upserted.closedReason ? { closedReason: upserted.closedReason } : {}), // The action the case now shows. Since ADR-074 d13 an UPDATED can be a next_action // change under an unchanged status; without it here the event would be // indistinguishable from the silent refresh it replaced. diff --git a/backend-ts/src/stores/store-contract.ts b/backend-ts/src/stores/store-contract.ts index ac3b8783..e05d13f4 100644 --- a/backend-ts/src/stores/store-contract.ts +++ b/backend-ts/src/stores/store-contract.ts @@ -797,6 +797,34 @@ export function caseStoreContract(label: string, freshStore: () => Promise { + const store = await freshStore(); + const outside = (over: Partial<{ subjectId: string }> = {}) => + store.upsertFromOutcome({ + runId: crypto.randomUUID(), + subjectId: over.subjectId ?? "emp-006", + measureId: "audiogram", + evaluationPeriod: "2026-06-13", + outcomeStatus: "MISSING_DATA", + outOfPopulation: true, + }); + assert.equal(await outside({ subjectId: "emp-009" }), null, "no case exists → nothing is opened"); + assert.equal((await store.listCases({})).length, 0); + + const opened = await upsert(store, "OVERDUE"); + assert.equal(opened?.status, "OPEN"); + const closed = await outside(); + assert.equal(closed?.disposition, "RESOLVED"); + assert.equal(closed?.status, "RESOLVED"); + assert.equal(closed?.closedReason, "OUT_OF_POPULATION"); + assert.equal(closed?.closedBy, null, "a SYSTEM closure"); + assert.equal(closed?.id, opened?.id, "the same row, closed — never a second case"); + assert.equal(await outside(), null, "the next nightly run is a no-op on the closed row: no CASE_UPDATED noise"); + const reopened = await upsert(store, "OVERDUE"); + assert.equal(reopened?.disposition, "REOPENED", "back in the population with a gap → the system closure reopens"); + assert.equal(reopened?.id, opened?.id); + }); + test(`[${label}] a rerun upserts the SAME case — never a duplicate (idempotency invariant)`, async () => { const store = await freshStore(); const first = await upsert(store, "OVERDUE"); From e50f51e7792743721d478e92f6e5f4ff1a9ab0a0 Mon Sep 17 00:00:00 2001 From: Taleef Date: Tue, 8 Sep 2026 13:45:38 -0400 Subject: [PATCH 4/6] fix(case): the out-of-population closure is gated on official routing, and rerun-to-verify closes such a case too Gemini review of the flip branch. The authored engine also sets inInitialPopulation (deriveInInitialPopulation emits it for every measure with a boolean Initial Population define), so ungated, an unenrolled audiogram subject would have stopped opening a case; the pipeline and rerun-to-verify now read the flag only where logicVersionFor says the official artifact ran, and tests pin the authored case on both paths. Rerun-to-verify closes an out-of-population case under OUT_OF_POPULATION as a system closure (closed_by NULL, reopenable), with a CASE_RESOLVED event. The gate workflow's routed default is the six-measure list, its verdict step fails the job on DO NOT FLIP while still uploading the report, the JSON evidence is asserted before upload, and the reproducibility check uses the job's working directory. --- .github/workflows/flip-gate.yml | 20 ++++++-- backend-ts/src/case/case-rerun.rates.test.ts | 48 +++++++++++++++++++ backend-ts/src/case/case-rerun.ts | 37 ++++++++++++-- backend-ts/src/run/run-pipeline.test.ts | 25 ++++++++++ backend-ts/src/run/run-pipeline.ts | 13 +++-- backend-ts/src/stores/case-store.ts | 3 +- .../src/wiring/official-flip-config.test.ts | 4 +- 7 files changed, 134 insertions(+), 16 deletions(-) diff --git a/.github/workflows/flip-gate.yml b/.github/workflows/flip-gate.yml index b8cb3ef4..cbfaca20 100644 --- a/.github/workflows/flip-gate.yml +++ b/.github/workflows/flip-gate.yml @@ -26,10 +26,10 @@ on: type: string default: "2026-12-31" routed: - description: "The deployment's CURRENT WORKWELL_OFFICIAL_MEASURES (the gate appends the measure under test)." + description: "The deployment's CURRENT WORKWELL_OFFICIAL_MEASURES (the gate appends the measure under test). Paste the value deploy-maui-mieweb.yml ships." required: true type: string - default: "cms122,cms125" + default: "cms122,cms125,cms2,cms130,cms165,cms137" permissions: contents: read @@ -91,8 +91,7 @@ jobs: fi # The vendored artifact must not have changed — only the gitignored sidecar is new. - name: The committed artifact is reproducible from its pin - working-directory: . - run: git diff --exit-code backend-ts/measures/official + run: git diff --exit-code measures/official - name: Gate shell: bash @@ -110,11 +109,24 @@ jobs: sort gate-stderr.log | uniq -c | sort -rn | head -40 echo "::endgroup::" ls -la .flip-gate/ + # The JSON is the evidence a flip PR attaches; the text report alone would let the upload + # below succeed with the evidence missing. + test -n "$(ls .flip-gate/*.json 2>/dev/null)" || { echo "::error::the gate wrote no .flip-gate/*.json"; exit 1; } - uses: actions/upload-artifact@v6 + if: always() with: name: flip-gate-${{ inputs.measure }}-${{ inputs.evaluation_date }} path: | backend-ts/.flip-gate/*.json backend-ts/gate-report.txt if-no-files-found: error + + # The CLI is DESCRIPTIVE and exits 0 on every verdict (ADR-072); in CI a green run over a "DO NOT + # FLIP" verdict would be read as evidence FOR. The report is uploaded either way; the job is red. + - name: The verdict is evidence FOR the flip + run: | + if grep -q "DO NOT FLIP" gate-report.txt; then + echo "::error::the gate's verdict is DO NOT FLIP — read gate-report.txt in the artifact" + exit 1 + fi diff --git a/backend-ts/src/case/case-rerun.rates.test.ts b/backend-ts/src/case/case-rerun.rates.test.ts index 8e0c2892..3cd4c93a 100644 --- a/backend-ts/src/case/case-rerun.rates.test.ts +++ b/backend-ts/src/case/case-rerun.rates.test.ts @@ -110,3 +110,51 @@ test("rerun-to-verify hands the action back to the SYSTEM, so the rate can keep "a computed action must say so, or patchCase's OPERATOR default freezes it", ); }); + +test("rerun-to-verify closes a case whose subject the OFFICIAL logic finds outside the initial population, as a system closure (ADR-078)", async () => { + const { rerunToVerify } = await import("./case-rerun.ts"); + const OUTSIDE = { + expressionResults: [], + official: { ecqmId: "CMS137", populationResults: rate(false).map((p) => ({ ...p, result: false })), measurementPeriod: { start: "2026-01-01", end: "2026-12-31" } }, + }; + let patched: Record | null = null; + const audits: Array<{ eventType: string; payload: Record }> = []; + const noop = async () => undefined; + const deps = { + cases: { + getCase: async () => existing, + patchCase: async (_id: string, patch: Record) => { patched = patch; return { ...existing, ...patch }; }, + listCases: async () => [existing], + upsertFromOutcome: noop, + countByLastRun: async () => 1, + }, + events: { + recordCaseEvent: noop, + appendAudit: async (input: { eventType: string; payload: Record }) => { audits.push({ eventType: input.eventType, payload: input.payload }); }, + caseTimeline: async () => [], + latestOutreachDeliveryStatus: async () => null, + }, + outcomes: { recordOutcome: noop, listOutcomes: async () => [] }, + runStore: { createRun: async () => ({ id: "run-rerun-oop" }), markRunning: noop, appendLog: noop, finalizeRun: noop, listRuns: async () => [] }, + engine: { + evaluate: async () => ({ outcome: "MISSING_DATA", evidence: OUTSIDE, inInitialPopulation: false }), + logicVersionFor: () => "official-fqm:1.0.000:artifact:terminology", + }, + } as unknown as RerunDeps; + + await rerunToVerify(deps, existing.id, "tester"); + const p = patched as Record | null; + assert.ok(p, "the case was patched"); + assert.equal(p!.status, "RESOLVED"); + assert.equal(p!.closedReason, "OUT_OF_POPULATION"); + assert.equal(p!.closedBy, null, "the system's determination, not the operator's closure — a later in-population outcome may reopen it"); + assert.ok(String(p!.nextAction).includes("not in the measure's initial population")); + assert.ok(audits.some((a) => a.eventType === "CASE_RESOLVED" && a.payload.closedReason === "OUT_OF_POPULATION"), "the closure is audited with its reason"); + + // The AUTHORED engine's flag does not close anything through this path either. + patched = null; + audits.length = 0; + const authored = { ...deps, engine: { evaluate: async () => ({ outcome: "MISSING_DATA", evidence: { expressionResults: [] }, inInitialPopulation: false }), logicVersionFor: () => "sha256:authored" } } as unknown as RerunDeps; + await rerunToVerify(authored, existing.id, "tester"); + assert.equal((patched as Record | null)?.status, "OPEN", "an authored out-of-population subject keeps the case open, unchanged"); +}); diff --git a/backend-ts/src/case/case-rerun.ts b/backend-ts/src/case/case-rerun.ts index 1e0ed76e..3454d251 100644 --- a/backend-ts/src/case/case-rerun.ts +++ b/backend-ts/src/case/case-rerun.ts @@ -26,6 +26,7 @@ import { compositeBundleSource } from "../wiring/subject-bundle-source.ts"; import { priorityFor, nextActionFor } from "./case-logic.ts"; import { toCaseDetail, type CaseDetail } from "./case-detail-read-model.ts"; import { caseRerunMeasurementPeriod } from "../run/run-period.ts"; +import { OFFICIAL_LOGIC_VERSION_PREFIX } from "../wiring/executor-router.ts"; export interface RerunDeps { cases: CaseStore; @@ -102,10 +103,19 @@ export async function rerunToVerify(deps: RerunDeps, caseId: string, actor: stri let verifiedStatus: string; let evidence: unknown; + // ADR-078: the OFFICIAL logic found the subject outside the initial population. The nightly run closes + // such a case under OUT_OF_POPULATION; a rerun-to-verify must reach the same answer, or the operator's + // own click leaves the case open until the next night (Gemini review). Authored measures set the flag + // too and are deliberately not read here — same gate as the pipeline. + let outOfPopulation = false; try { const result = await deps.engine.evaluate({ measureId: existing.measureId, patientBundle: bundle, evaluationDate: evalDate }); verifiedStatus = result.outcome; evidence = result.evidence; + const logicVersionFor = (deps.engine as { logicVersionFor?: (measureId: string) => string | undefined }).logicVersionFor; + outOfPopulation = + result.inInitialPopulation === false && + (logicVersionFor?.(existing.measureId)?.startsWith(OFFICIAL_LOGIC_VERSION_PREFIX) ?? false); } catch (err) { verifiedStatus = "MISSING_DATA"; evidence = { evaluationError: "engine failure", message: String((err as Error)?.message ?? err) }; @@ -121,12 +131,18 @@ export async function rerunToVerify(deps: RerunDeps, caseId: string, actor: stri evidence, }); - const updatedCaseStatus = verificationCaseStatus(existing.status, verifiedStatus); - const nextAction = verificationNextAction(verifiedStatus, existing.measureId, evidence); - const closing = isClosing(verifiedStatus); + const updatedCaseStatus = outOfPopulation ? "RESOLVED" : verificationCaseStatus(existing.status, verifiedStatus); + const nextAction = outOfPopulation + ? "No follow-up needed: not in the measure's initial population on verification rerun." + : verificationNextAction(verifiedStatus, existing.measureId, evidence); + const closing = isClosing(verifiedStatus) || outOfPopulation; const closedAt = closing ? new Date().toISOString() : null; - const closedReason = verifiedStatus === "COMPLIANT" ? "RERUN_VERIFIED" : verifiedStatus === "EXCLUDED" ? "RERUN_EXCLUDED" : null; - const closedBy = closing ? actor : null; + const closedReason = + verifiedStatus === "COMPLIANT" ? "RERUN_VERIFIED" : verifiedStatus === "EXCLUDED" ? "RERUN_EXCLUDED" : outOfPopulation ? "OUT_OF_POPULATION" : null; + // An out-of-population closure is the SYSTEM's determination even when a person clicked rerun: it + // stays `closed_by = NULL` so a later in-population outcome reopens it, exactly as the nightly path + // closes it. A verified COMPLIANT/EXCLUDED is the operator's closure and keeps their name. + const closedBy = closing && !outOfPopulation ? actor : null; const actionPayload = { priorOutcomeStatus: existing.currentOutcomeStatus, @@ -197,6 +213,17 @@ export async function rerunToVerify(deps: RerunDeps, caseId: string, actor: stri refMeasureVersionId: existing.measureId, payload: { ...verificationPayload, exclusionReason: "Excluded on verification rerun." }, }); + } else if (outOfPopulation) { + await deps.events.appendAudit({ + eventType: "CASE_RESOLVED", + entityType: "case", + entityId: caseId, + actor, + refRunId: run.id, + refCaseId: caseId, + refMeasureVersionId: existing.measureId, + payload: { ...verificationPayload, closedReason: "OUT_OF_POPULATION", summary: "Case closed by rerun-to-verify: the subject is outside the measure's initial population (ADR-078).", runId: run.id }, + }); } await deps.runStore.finalizeRun(run.id, hasEvaluationError ? "PARTIAL_FAILURE" : "COMPLETED"); diff --git a/backend-ts/src/run/run-pipeline.test.ts b/backend-ts/src/run/run-pipeline.test.ts index 14b7e72f..ca17969d 100644 --- a/backend-ts/src/run/run-pipeline.test.ts +++ b/backend-ts/src/run/run-pipeline.test.ts @@ -1599,3 +1599,28 @@ test("ADR-078: a subject the official logic finds OUTSIDE the initial population assert.equal(captured.filter((e) => e.eventType === "CASE_RESOLVED").length, 1, "the closure is audited"); assert.equal(captured.filter((e) => e.eventType === "CASE_CREATED").length, 0); }); + +test("ADR-078 is gated on official routing: an AUTHORED measure's out-of-population subject still opens a case", async () => { + // `deriveInInitialPopulation` emits the flag for authored measures too (their CQL has a boolean + // `Initial Population` define). Not enrolled in a hearing conservation program is a workflow fact whose + // case handling ADR-078 leaves alone — the closure is for the PUBLISHED logic's population only. + const db = await createSqliteD1(join(tmpdir(), `workwell-pipeline-oop-authored-${crypto.randomUUID()}.sqlite`)); + await db.exec(RUN_STORE_FLOOR_DDL.replace(/\n/g, " ")); + const caseStore = new SqliteCaseStore(db); + const deps: RunPipelineDeps = { + runStore: new SqliteRunStore(db), + outcomeStore: new SqliteOutcomeStore(db), + caseStore, + engine: { + evaluate: async () => ({ outcome: "MISSING_DATA", evidence: { expressionResults: [{ define: "Initial Population", result: false }] }, inInitialPopulation: false }), + logicVersionFor: () => "sha256:authored", + } as unknown as RunPipelineDeps["engine"], + employees: EMPLOYEES.slice(0, 1), + actor: "cm@workwell.dev", + events: { async appendAudit() {} }, + }; + await executeManualRun(deps, { scopeType: "MEASURE", measureId: "audiogram", triggeredBy: "test" }); + const cases = await caseStore.listCases({ limit: 10 }); + assert.equal(cases.length, 1, "the authored engine's flag does not suppress the case"); + assert.equal(cases[0]!.status, "OPEN"); +}); diff --git a/backend-ts/src/run/run-pipeline.ts b/backend-ts/src/run/run-pipeline.ts index 213d5610..f009b487 100644 --- a/backend-ts/src/run/run-pipeline.ts +++ b/backend-ts/src/run/run-pipeline.ts @@ -870,10 +870,15 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun (await deps.engine.evaluate({ measureId: item.measureId, patientBundle: bundle, evaluationDate: evalDate })); status = result.outcome; evidence = result.evidence; - // Read off the executor's own flag, not re-derived from evidence: the authored engine never - // sets it, so an authored MISSING_DATA keeps opening a case (it means "no record"), while an - // official `false` means the logic ran and the subject is not the measure's concern (ADR-078). - outOfPopulation = result.inInitialPopulation === false; + // Read off the executor's own flag, and ONLY for an officially routed measure (ADR-078). The + // authored engine sets the flag too — `deriveInInitialPopulation` emits it for every measure + // with a boolean `Initial Population` define, which is all of them (see the ADR-043 gate below + // for the same lesson) — but an authored "not in the initial population" is a workflow fact + // (not enrolled in the hearing conservation program) whose case handling is unchanged. An + // official `false` means the published logic ran and the subject is not the measure's concern. + outOfPopulation = + result.inInitialPopulation === false && + (deps.engine.logicVersionFor?.(item.measureId)?.startsWith(OFFICIAL_LOGIC_VERSION_PREFIX) ?? false); // ADR-043 — record membership from the FINAL outcome, whichever path produced it (batch prefetch // or the individual fallback on this line). Reading it here rather than in the pre-pass is what // makes the roster complete before it is judged. A failed evaluation lands in `catch` below and diff --git a/backend-ts/src/stores/case-store.ts b/backend-ts/src/stores/case-store.ts index c5280c60..116c6760 100644 --- a/backend-ts/src/stores/case-store.ts +++ b/backend-ts/src/stores/case-store.ts @@ -37,7 +37,8 @@ export interface UpsertCaseInput { /** * The official logic evaluated this subject and found them OUTSIDE the initial population (ADR-078). * Never creates a case; resolves an active one with `closed_reason='OUT_OF_POPULATION'` (a system - * closure). The run pipeline sets it from the persisted official evidence; the status stays MISSING_DATA. + * closure). The run pipeline sets it from the executor's own `inInitialPopulation: false`, and only + * for an OFFICIALLY routed measure; the persisted status stays MISSING_DATA. */ outOfPopulation?: boolean; } diff --git a/backend-ts/src/wiring/official-flip-config.test.ts b/backend-ts/src/wiring/official-flip-config.test.ts index 4838fa32..b1ea16f3 100644 --- a/backend-ts/src/wiring/official-flip-config.test.ts +++ b/backend-ts/src/wiring/official-flip-config.test.ts @@ -319,8 +319,8 @@ test("PR-9c: the shipped configuration constructs cleanly — no routing problem // // `complete` must read BOTH conditions. Reading `truncated` alone was true-but-narrow: it means "the // sidecar holds every code the bundle DECLARED", which says nothing about a value set the bundle - // never declared at all (ADR-053). No shipped measure has one today — all five vendored artifacts - // ship every value set their ELM retrieves — so this changes no verdict now; it stops the predicate + // never declared at all (ADR-053). No shipped measure has one today — every vendored artifact + // ships every value set its ELM retrieves — so this changes no verdict now; it stops the predicate // silently meaning less than its name the first time one does. const complete = ALL_SHIPPED.every((id) => { const artifact = loadOfficialArtifact(id); From 4608a167e50682975e2b5589bbb668c90ef11d52 Mon Sep 17 00:00:00 2001 From: Taleef Date: Tue, 8 Sep 2026 13:51:15 -0400 Subject: [PATCH 5/6] fix(run): an out-of-population outcome always reaches the case store Codex review of #542: the upsert was gated on the active-case preload, so a transient failure of that read (caught, leaving the set empty) would have left every out-of-population case open for another night. planCaseUpsert is a no-op where no row exists, so the upsert is unconditional, like COMPLIANT; EXCLUDED stays gated because its no-case branch inserts. --- backend-ts/src/run/run-pipeline.ts | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/backend-ts/src/run/run-pipeline.ts b/backend-ts/src/run/run-pipeline.ts index f009b487..28aaa0b5 100644 --- a/backend-ts/src/run/run-pipeline.ts +++ b/backend-ts/src/run/run-pipeline.ts @@ -968,16 +968,16 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun // with NO existing case would INSERT a new EXCLUDED case and re-pollute the gate. The outcome above // is ALWAYS persisted (CQL is the sole compliance authority — ADR-008). Empty/absent segments ⇒ all. const caseKey = `${item.employee.externalId}|${item.measureId}|${period}`; - // (3) OUT OF THE INITIAL POPULATION (ADR-078) — the same close-only shape as EXCLUDED: a subject the - // official logic found outside the measure's population never opens a case, and an active one (they - // were in the population last period, or were carded before this rule) is closed by the system - // under `OUT_OF_POPULATION`. Until 2026-09-08 this was ADR-043's recorded fan-out: every + // (3) OUT OF THE INITIAL POPULATION (ADR-078) — close-only like COMPLIANT, and like COMPLIANT it is + // NOT gated on the active-case snapshot: `planCaseUpsert` returns a no-op where no row exists, so + // the upsert is always safe, and gating it on `activeCaseKeys` would mean a transient failure of + // that preload (caught above, leaving the set empty) silently left every out-of-population case + // open for another night (Codex review, #542). EXCLUDED stays gated because its no-case branch + // INSERTS. A subject the official logic found outside the measure's population never opens a case, + // and an active one (in the population last period, or carded before this rule) is closed by the + // system under `OUT_OF_POPULATION`. Until 2026-09-08 this was ADR-043's recorded fan-out: every // non-diabetic opened a MEDIUM CMS122 case, and six routed measures made that the worklist. - const closeOnly = - status === "COMPLIANT" || - (status === "EXCLUDED" && activeCaseKeys.has(caseKey)) || - (outOfPopulation && activeCaseKeys.has(caseKey)); - const outOfPopulationWithNoCase = outOfPopulation && !activeCaseKeys.has(caseKey); + const closeOnly = status === "COMPLIANT" || (status === "EXCLUDED" && activeCaseKeys.has(caseKey)) || outOfPopulation; // Live WebChart subjects are display-applicable (their roster cells show real chips) but must NOT // OPEN cases: rerun-to-verify returns a non-mutating 409 for `wc|` subjects until fetch-one-patient // lands, so a newly-created wc case would be un-closeable. Case CREATION eligibility is therefore @@ -1013,7 +1013,7 @@ export async function finishManualRun(deps: RunPipelineDeps, planned: PlannedRun if (gatedBySegment.sites.size < 12) gatedBySegment.sites.add(item.employee.site ?? "(no site)"); gatedBySegment.measures.add(item.measureId); } - if (deps.caseStore && !outOfPopulationWithNoCase && (closeOnly || (!isLiveWebChartSubject && segmentApplicable()))) { + if (deps.caseStore && (closeOnly || (!isLiveWebChartSubject && segmentApplicable()))) { const upserted = await deps.caseStore.upsertFromOutcome({ runId: runId, subjectId: item.employee.externalId, From a7b6bad0fa086c79a692f2acd76f2694f997f47f Mon Sep 17 00:00:00 2001 From: Taleef Date: Tue, 8 Sep 2026 14:00:24 -0400 Subject: [PATCH 6/6] =?UTF-8?q?docs(evidence):=20cms130=20and=20cms165=20g?= =?UTF-8?q?ated=20over=20the=20whole=20Maui=20roster=20=E2=80=94=2064/64?= =?UTF-8?q?=20and=2068/68,=209,257=20and=206,837=20in=20the=20initial=20po?= =?UTF-8?q?pulation,=200=20errors?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both run through the credentialed flip-gate workflow (runs 34258656822 and 34258659692). The workflow now copies the gate JSON out of .flip-gate/ before upload: the artifact glob does not descend into a dot-directory, so the first two runs uploaded only the text report. --- .github/workflows/flip-gate.yml | 7 +++-- docs/DECISIONS.md | 5 ++-- docs/MEASURES.md | 6 +++-- docs/evidence/FLIP_GATE_2026-09-08_CMS130.md | 24 +++++++++++++++++ docs/evidence/FLIP_GATE_2026-09-08_CMS165.md | 28 ++++++++++++++++++++ 5 files changed, 64 insertions(+), 6 deletions(-) create mode 100644 docs/evidence/FLIP_GATE_2026-09-08_CMS130.md create mode 100644 docs/evidence/FLIP_GATE_2026-09-08_CMS165.md diff --git a/.github/workflows/flip-gate.yml b/.github/workflows/flip-gate.yml index cbfaca20..31b54b45 100644 --- a/.github/workflows/flip-gate.yml +++ b/.github/workflows/flip-gate.yml @@ -110,15 +110,18 @@ jobs: echo "::endgroup::" ls -la .flip-gate/ # The JSON is the evidence a flip PR attaches; the text report alone would let the upload - # below succeed with the evidence missing. + # below succeed with the evidence missing. Copied out of the dot-directory first: the first + # two runs of this workflow uploaded only gate-report.txt, because the artifact glob does not + # descend into `.flip-gate/`. test -n "$(ls .flip-gate/*.json 2>/dev/null)" || { echo "::error::the gate wrote no .flip-gate/*.json"; exit 1; } + mkdir -p flip-gate-out && cp .flip-gate/*.json flip-gate-out/ - uses: actions/upload-artifact@v6 if: always() with: name: flip-gate-${{ inputs.measure }}-${{ inputs.evaluation_date }} path: | - backend-ts/.flip-gate/*.json + backend-ts/flip-gate-out/*.json backend-ts/gate-report.txt if-no-files-found: error diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index a9f7c247..249ab338 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -46,8 +46,9 @@ been the whole worklist. and `reconcile-maui-mieweb.yml`, which must agree). Evidence attached to the flip: cms2's whole-roster gate (36/36, 17,795/20,000 in the initial population, 5,413 actionable, 0 errors), cms137's (`docs/evidence/FLIP_GATE_2026-09-07_CMS137.md`), the cross-engine sweeps, and the credentialed - gates for cms130 and cms165 run through the new `flip-gate.yml` workflow, because their pinned - sidecars are VSAC-completed and do not resolve locally. + gates for cms130 (64/64, 9,257 in the initial population, 4,997 actionable) and cms165 (68/68, + 6,837 and 2,431), run through the `flip-gate.yml` workflow because their pinned sidecars are + VSAC-completed and do not resolve locally (`FLIP_GATE_2026-09-08_CMS130.md`, `_CMS165.md`). 2. **A subject the executor finds outside the initial population never opens a case.** The pipeline reads the executor's own `inInitialPopulation: false` — never re-derived from evidence, so an authored MISSING_DATA ("no record") keeps opening cases — and treats it as a close-only outcome: diff --git a/docs/MEASURES.md b/docs/MEASURES.md index 1ba68e2e..312f27cc 100644 --- a/docs/MEASURES.md +++ b/docs/MEASURES.md @@ -26,8 +26,10 @@ six** (ADR-078, owner decision): cms122, cms125, cms2, cms130, cms165 and cms137 CMS165 and CMS137 are Active, gated AND executing there; TWH still routes cms122 and cms125 only. All four pass their full MADiE deck against the runtime (36/36, 64/64, 68/68 and 45/45, zero errors); cms2's whole-roster gate reads 17,795 of 20,000 in the initial population and 5,413 actionable -(`docs/evidence/FLIP_GATE_2026-09-08_CMS2.md`), and the credentialed gates for cms130 and cms165 run -through `.github/workflows/flip-gate.yml`. **CMS137 is multi-rate** (Initiation and Engagement, +(`docs/evidence/FLIP_GATE_2026-09-08_CMS2.md`); cms130's, run through the credentialed +`.github/workflows/flip-gate.yml`, reads 64/64 and 9,257 in the initial population with 4,997 actionable +(`FLIP_GATE_2026-09-08_CMS130.md`), and cms165's 68/68 and 6,837 with 2,431 actionable +(`FLIP_GATE_2026-09-08_CMS165.md`) — on the sandbox corpus, which is all it claims. **CMS137 is multi-rate** (Initiation and Engagement, ADR-074): one catalog row, one outcome per patient reduced to the worst rate, and every rate persisted and exported — and the roster cell, the case detail, the case's next action and the CDS card name the rate the patient actually missed ("without treatment initiation within 14 days" or "initiated but not engaged diff --git a/docs/evidence/FLIP_GATE_2026-09-08_CMS130.md b/docs/evidence/FLIP_GATE_2026-09-08_CMS130.md new file mode 100644 index 00000000..a4e3a5fe --- /dev/null +++ b/docs/evidence/FLIP_GATE_2026-09-08_CMS130.md @@ -0,0 +1,24 @@ +# CMS130's flip gate over the whole Maui roster + +Date: 2026-09-08. Run where the credentialed terminology sidecar resolves — the `flip-gate.yml` +workflow (run 34258656822), which vendors CMS130's sidecar completed from VSAC, then: + +```bash +WORKWELL_INSTANCE=maui WORKWELL_MAUI_CORPUS_SIZE=20000 WORKWELL_OFFICIAL_MEASURES=cms122,cms125 \ + pnpm flip-gate --measure cms130 --evaluation-date 2026-12-31 --subjects all +``` + +| reading | result | +|---|---| +| MADiE — the steward's own expected vectors | **64 of 64** agree, 0 disagree | +| Roster — the artifact over the deployment's own 20,000 corpus patients, routed as `cms122,cms125,cms130` | **9,257 in the initial population and denominator**, 4,997 actionable, **0 evaluation errors** | +| Outcome distribution | OVERDUE 4,997 · COMPLIANT 3,815 · EXCLUDED 445 · MISSING_DATA 10,743 | +| `effectivePeriod` | covers the measured year (2026) | + +Verdict printed by the gate: *the three readings agree — evidence FOR the flip; the flip itself is a +workflow edit a human still makes.* The 10,743 MISSING_DATA are subjects outside the initial population +(outside the 45–75 age band, or no qualifying encounter), shown on the roster as OUT_OF_POPULATION and +opening no case (ADR-077 d7, ADR-078 d2). + +Second-engine evidence for the same measure: `CROSS_ENGINE_2026-09-07_CMS130_CMS165.md` (63 of 64 +agree; the one disagreement carries the medication-period signature diagnosed for CMS2). diff --git a/docs/evidence/FLIP_GATE_2026-09-08_CMS165.md b/docs/evidence/FLIP_GATE_2026-09-08_CMS165.md new file mode 100644 index 00000000..39160e84 --- /dev/null +++ b/docs/evidence/FLIP_GATE_2026-09-08_CMS165.md @@ -0,0 +1,28 @@ +# CMS165's flip gate over the whole Maui roster + +Date: 2026-09-08. Run where the credentialed terminology sidecar resolves — the `flip-gate.yml` +workflow (run 34258659692), which vendors CMS165's sidecar completed from VSAC, then: + +```bash +WORKWELL_INSTANCE=maui WORKWELL_MAUI_CORPUS_SIZE=20000 WORKWELL_OFFICIAL_MEASURES=cms122,cms125 \ + pnpm flip-gate --measure cms165 --evaluation-date 2026-12-31 --subjects all +``` + +| reading | result | +|---|---| +| MADiE — the steward's own expected vectors | **68 of 68** agree, 0 disagree | +| Roster — the artifact over the deployment's own 20,000 corpus patients, routed as `cms122,cms125,cms165` | **6,837 in the initial population and denominator**, 2,431 actionable, **0 evaluation errors** | +| Outcome distribution | COMPLIANT 4,026 · OVERDUE 2,431 · EXCLUDED 380 · MISSING_DATA 13,163 | +| `effectivePeriod` | covers the measured year (2026) | + +Verdict printed by the gate: *the three readings agree — evidence FOR the flip; the flip itself is a +workflow edit a human still makes.* The 13,163 MISSING_DATA are subjects outside the initial population +(no essential hypertension diagnosis in the window, or outside the age band), shown on the roster as +OUT_OF_POPULATION and opening no case (ADR-077 d7, ADR-078 d2). + +**What this does and does not say.** The roster reading runs with `trustMetaProfile` (ADR-076 d1) over a +corpus that stamps every profile the artifact retrieves, and with the blood-pressure stamp from #539, so it +is evidence that cms165 works on the SANDBOX. It says nothing about WebChart data, where the ingest half +of #533 (every QI-Core profile stamped at ingest, BP panels arriving `final` rather than `unknown`) still +gates the PHI phase. cms165's cross-engine comparison remains an open question (#532), and no +cross-engine number is quoted for it.