C3: Failure inputs in the export (#30, #31)

CONTEXT.md

@@ -175,7 +175,7 @@ The best-case assumption that water consumed on a failure day is a mixture of wa
 _Avoid_: mixed concentration, dilution, storage effect
 
 **Combined failure**:
-Failure events in two or more treatment steps on the same day. Worst-case loses all their LRVs for the whole day. Under the mixed-water assumption the failure events are arranged within the day so that the mixed water has the lowest pathogen concentration. This means they do not overlap if their durations fit into one day, and otherwise overlap only by the minutes that exceed a day, shared by the steps whose joint loss matters least.
+Failure events in two or more treatment steps on the same day. Worst-case loses all their positive LRVs for the whole day. Under the mixed-water assumption the failure events do not overlap if their durations fit into one day. Two failure events longer than a day in total overlap only by the minutes that exceed a day. Three or more failure events longer than a day in total count like worst-case: all these steps are down for the whole day. Only steps that lose LRV for the pathogen group count, so the same day can count like worst-case for one reference pathogen and not for another.
 
 **Next phase**:
 Capabilities identified for future development and not present in the application today.

docs/QMRA_Failure_Approach.md

@@ -109,17 +109,19 @@ is recorded in `docs/adr/0001-failure-days-drawn-per-exposure-event.md`.
    the existing Monte Carlo simulation. The section-3 approach is implemented directly;
    Eq. 6 is not implemented as a separate path, not even for a single failing step. See
    ADR-0001.
-6. **Combined failures in best-case.** Eq. 5 is extended so that failure events of
-   different steps on the same day overlap as little as possible: not at all if their
-   durations fit into 1,440 minutes, otherwise only by the minutes beyond a full day.
+6. **Combined failures in best-case.** Eq. 5 is extended to failure events of different
+   steps on the same day: they do not overlap if their durations fit into 1,440 minutes,
+   and two steps longer than a day in total overlap only by the minutes beyond a full day.
    Example (best-case LRV 10 = UV 4 + UF 4 + other steps 2): UV down 60 min and UF down
    120 min give 4.2 % of the day at LRV 6, 8.3 % at LRV 6 and 87.5 % at LRV 10, a mixed
    LRV of about 6.9. UV and UF each down 900 min overlap by 360 min: 25 % at LRV 2 and
    2 × 37.5 % at LRV 6, a mixed LRV of about 2.6. Worst-case loses all failing steps'
-   LRVs for the whole day. With three or more failing steps, the unavoidable overlap is
-   placed on the steps whose joint loss gives the lowest mixed concentration; in general,
-   best-case arranges the failure events of a day so that the mixed water has the lowest
-   concentration, which always implies minimal overlap (decided 2026-09-24).
+   LRVs for the whole day. If three or more steps fail on the same day and their
+   durations add up to more than 1,440 minutes, best-case counts that day like worst-case:
+   all these steps are down for the whole day. Only steps that lose LRV for the pathogen
+   group count (decided 2026-09-25, replacing the
+   lowest-concentration arrangement of 2026-09-24, which needed a numerical solver for a
+   rare exception).
 7. **Results.** Only the result including failures is shown, and the reference-level
    exceedance is taken on it. The effect of failures is seen by comparing scenarios with
    different failure inputs in the assessment comparison view. That view compares annual

docs/failure_roadmap.md

@@ -21,7 +21,7 @@ at least one positive LRV accept failure inputs (D4); values of a personal treat
 carried into the assessment; saving, reopening and editing keep them. Existing saved
 assessments get failure frequency 0, so their results are unchanged.
 
-Not included: any effect on the calculation (C2), export and report (C3), explanations
+Not included: any effect on the calculation (C2), the export (C3), explanations
 beyond field labels and units (C4).
 
 ### C2 Failure calculation — Large
@@ -31,25 +31,29 @@ Spec issue: #___ (draft: `docs/specs/c2-failure-calculation.md`)
 Purpose: results include treatment failures as decided in ADR-0001.
 
 Scope: failure days drawn per exposure event and per treatment step inside the Monte Carlo
-simulation; worst-case loses the full LRV of every failing step; best-case applies the
+simulation; worst-case loses every positive LRV of every failing step; best-case applies the
 mixed-water assumption; a failure removes only positive LRVs, LRVs of 0 or below stay
-(D4); combined failures overlap as little as possible; the result page
+(D4); combined failures that fit into a day do not overlap, two steps longer than a day
+overlap only by the extra minutes, and three or more steps longer than a day in total count
+like worst-case; the result page
 and the reference-level exceedance show the result including failures. With every failure
 frequency at 0, results equal today's. Tested with checks that need no external expected
 values: failure frequency 0 reproduces today's results exactly, worst-case risk is never
 below best-case, a higher failure frequency never lowers the risk, LRVs of 0 or below are
-unaffected, and combined failures overlap minimally in best-case.
+unaffected, and combined failures follow the best-case rules for overlap.
 
 Not included: the comparison with an independent calculation (C5), showing normal
 operation next to the result with failures (decided against), DALYs in the assessment
 comparison (later).
 
-### C3 Failure inputs in export and report — Small
+### C3 Failure inputs in the export — Small
 
-The export package's treatment table and the HTML report show each treatment step's failure
-frequency and failure duration, so a reader can see which failures a result is based on.
-Left out: renaming export files (`inflows.csv` → `inflow_concentration.csv` waits until
-later), origin and reference columns for inputs, changes to the assessment comparison.
+The export package's treatment table shows each treatment step's failure frequency and
+failure duration, so a reader can see which failures a result is based on. Left out: the
+HTML report, which shows no inputs at all and so doesn't show failure inputs either
+(decided 2026-09-25); renaming export files (`inflows.csv` → `inflow_concentration.csv`
+waits until later); origin and reference columns for inputs; changes to the assessment
+comparison.
 
 ### C4 Explain treatment failure — Small
 

docs/progress.md

@@ -0,0 +1,101 @@
+# Progress: treatment failure (C1–C5)
+
+Status: 2026-09-29. Work of 2026-09-24 to 2026-09-28 on the Changes in `docs/failure_roadmap.md`.
+Nothing is merged into `main` (roadmap decision D5): each Change is a draft pull request that
+targets `main` only so that the pipeline deploys it to dev. The production jobs run only for
+`main` itself.
+
+## Overview
+
+| Change | Branch (based on) | Pull request | Tickets | State |
+|---|---|---|---|---|
+| C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`) | Built, tested manually on dev |
+| C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | Built, tested manually on dev |
+| C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open |
+| C4 Explain treatment failure | not created yet (will be based on C2) | — | #32, #33 (`12`, `13`) | Tickets ready |
+| C5 Failure benchmark | — | — | — | Waits for decisions D1–D3 |
+
+**Dev** (https://dev.qmra.org) runs C1 + C2 + C3 (PR #34, commit `27b5268`) since 2026-09-25.
+
+**Review of 2026-09-28** (by a teammate):
+- **Statuses:** tickets 04–11 are set to "review".
+- **Review notes:** ticket 09 records a review of the C2 commits against the Spec, ADR-0001 and the code; tickets 10 and 11 record a review of the C3 commits.
+- **D4 wording:** `CONTEXT.md`, the roadmap and the vision now say that worst-case loses every *positive* LRV of a failing step, as the code does.
+
+## What each Change delivers
+
+### C1 Failure inputs (#28)
+
+- **Configurator (#21, `57bdd21`):**
+  - each treatment step has a failure frequency (days per year, 0–365, default 0) and a failure duration (whole minutes, 1–1440, default 30), with units;
+  - values are checked on run and save, and stay with their step through save, reopen and edit;
+  - the migration gives existing assessments 0 and 30;
+  - one shared validation rule replaces the duplicated min ≤ max LRV check.
+- **Positive LRV and guests (#22, `52e6087`):**
+  - a frequency above 0 needs at least one positive LRV (D4);
+  - guests now see validation messages on the affected field and step (before, an invalid guest result failed silently);
+  - the treatments section grows with its content instead of hiding the new fields under the footer.
+- **Personal treatment steps (#23, `7f05bcd`):** they store both values under the same rules, the configurator copies them into a newly added step, and the copy can be changed without touching the personal step.
+
+### C2 Failure calculation (#29)
+
+- **Tidy-up (#24, `93c8f76`):** the simulation works per exposure event. Results are bit-identical to before (checked on 28 scenarios, 1932 values).
+- **Failure days and worst-case (#25, `2f6efec`):**
+  - each exposure event falls on a failure day of each step with probability frequency / 365 (ADR-0001);
+  - the draws come from their own fixed seed, one stream per step position, shared by both cases and all pathogens;
+  - worst-case loses the failing steps' positive minimum LRVs for the whole day.
+- **Best-case (#26, `a8a1972`):** the mixed-water assumption. For combined failures of one day, only steps that lose LRV for the pathogen group count:
+  - failures that fit into a day don't overlap (one step: Eq. 5);
+  - two steps longer than a day overlap by the extra minutes;
+  - three or more steps longer than a day in total count like worst-case.
+- **Without failures:** results are exactly those from before C2.
+- **Local timing** (drinking water, 3 pathogens, 5 bundled steps): 0.07 s without failures, 0.13–0.19 s with failures.
+
+### C3 Failure inputs in the export (#34)
+
+- **Reference export test (#30, `7085d78`, fixes `305ec0f`, `27b5268`):** implements the "Export stability" Quality goal.
+  - A fixed benchmark assessment is exported and compared with `qmra/risk_assessment/tests/reference_export/`: the same files, identical text, and each number within 0.1 % relative. Plots are only checked for presence.
+  - `UPDATE_REFERENCE_EXPORT=1` rewrites the reference on purpose (see the README there).
+- **Failure columns (#31, `a04e2f9`):** `treatments.csv` has "Failure frequency (days per year)" and "Failure duration (minutes)", repeated on each step's three rows. The HTML report stays without inputs.
+- **For the release notes:** `treatments.csv` has two new columns at the end.
+
+## Decisions made during implementation
+
+All on 2026-09-25 unless noted. The documents named are updated.
+
+- **C1: failure fields are required in the form.** Existing form tests got the two fields in their input data.
+- **C1: guests see validation errors.** The guest result returns the messages, and the configurator shows them on the affected step (option a).
+- **C1: dev only moves forward.** The migrations can't be reversed on dev, and only C1 and branches built on it go there until the release. The columns get no database default.
+- **C2: statistics instead of the mean.** Tests check the returned statistics and the exceedance, because the mean isn't returned. Per-year coupling makes that equivalent.
+- **C2: one C1 test split.** With frequency 0, results and export stay identical. With failures, only values change.
+- **C2: three or more steps over a day count like worst-case in best-case.** This replaces the lowest-concentration arrangement decided on 2026-09-24, which needed a linear program whose solver failed for large combined losses; the per-pathogen arrangement rule is dropped with it. See the C2 Spec, `CONTEXT.md` (*Combined failure*), decision 6 in `docs/QMRA_Failure_Approach.md`, the roadmap and the vision.
+- **C2: only steps that lose LRV for the pathogen group count** towards "three or more" (Reading 1).
+- **C2: tested on dev with a PR into `main`,** because the pipeline deploys only pull requests into `main`.
+- **C3 and C4 have no Spec** (Small Changes). The Tickets follow the roadmap entries.
+- **C3: the HTML report shows no inputs** and doesn't get failure inputs. Roadmap entry corrected.
+- **C3: failure columns are repeated on each step's three rows** of `treatments.csv`.
+- **C3: a reference export test is added**, comparing numbers within 0.1 %. CI's processor and math library differ from this server's by up to 10⁻⁵ relative in tiny best-case probabilities.
+- **C4: texts are drafted by Claude** and reviewed by a domain expert before the release. Step 4 of the guided tour is extended, and a new Sphinx page is added.
+- **C3 and C4 are based on C2,** so dev keeps C1 + C2 while they are tested.
+
+## Open items
+
+**Checks still to do**
+- **C1 (#28):** real-data check; "I can explain this Change".
+- **C2 (#29):** calculation time measured on dev and recorded in the PR (ticket 09, #26); real-data check; "I can explain this Change".
+- **C3 (#34):** manual check on dev, from the list in the PR; real-data check; "I can explain this Change".
+
+**Work not started**
+- **C4:** #32 (FAQ and guided tour) and #33 (Sphinx page) can start now.
+- **C5:** needs decisions D1 (tolerance), D2 (who writes the independent script) and D3 (who signs off). The script must implement the combined-failure rule of 2026-09-25.
+
+**Known issues and risks**
+- **Bug #27** (older than C1): an empty "Events per year" or "Volume per event" gives a server error for guests and registered users.
+- **Lost click** (existing behaviour): typing into a field and clicking "Show assessment results" straight away loses the first click.
+- **Precision:** tiny best-case probabilities lose precision in `1 − exp(−k·dose)`. Making the calculation more precise (`expm1`/`log1p`) would be a separate change to the calculation.
+- **Step order:** failure draws are keyed by a step's position in the train. Saved assessments pass their steps without an explicit order (stable in practice).
+- **Unpinned dependencies:** numpy and pandas aren't pinned; reproducibility across releases is an open question for Architecture.
+- **Housekeeping:**
+  - the C1 and C2 Spec issues were never created (`#___` in the roadmap);
+  - the local branch `c2-failure-calculation` on the Controlled server is behind GitHub (the review commits of 2026-09-28) and holds one ticket commit (`5ca99b1`) that is on GitHub only through `c3-failure-inputs-in-export`. Update it from GitHub before working on it again.
+  - The GitHub command-line login on the Controlled server has expired (`HTTP 401`); issues and pull requests can't be updated from there until someone logs in again.