diff --git a/CONTEXT.md b/CONTEXT.md index f44d7be..5c82737 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -175,7 +175,7 @@ The best-case assumption that water consumed on a failure day is a mixture of wa _Avoid_: mixed concentration, dilution, storage effect **Combined failure**: -Failure events in two or more treatment steps on the same day. Worst-case loses all their LRVs for the whole day. Under the mixed-water assumption the failure events are arranged within the day so that the mixed water has the lowest pathogen concentration. This means they do not overlap if their durations fit into one day, and otherwise overlap only by the minutes that exceed a day, shared by the steps whose joint loss matters least. +Failure events in two or more treatment steps on the same day. Worst-case loses all their positive LRVs for the whole day. Under the mixed-water assumption the failure events do not overlap if their durations fit into one day. Two failure events longer than a day in total overlap only by the minutes that exceed a day. Three or more failure events longer than a day in total count like worst-case: all these steps are down for the whole day. Only steps that lose LRV for the pathogen group count, so the same day can count like worst-case for one reference pathogen and not for another. **Next phase**: Capabilities identified for future development and not present in the application today. diff --git a/docs/QMRA_Failure_Approach.md b/docs/QMRA_Failure_Approach.md index 8dd2dec..e80458a 100644 --- a/docs/QMRA_Failure_Approach.md +++ b/docs/QMRA_Failure_Approach.md @@ -109,17 +109,19 @@ is recorded in `docs/adr/0001-failure-days-drawn-per-exposure-event.md`. the existing Monte Carlo simulation. The section-3 approach is implemented directly; Eq. 6 is not implemented as a separate path, not even for a single failing step. See ADR-0001. -6. **Combined failures in best-case.** Eq. 5 is extended so that failure events of - different steps on the same day overlap as little as possible: not at all if their - durations fit into 1,440 minutes, otherwise only by the minutes beyond a full day. +6. **Combined failures in best-case.** Eq. 5 is extended to failure events of different + steps on the same day: they do not overlap if their durations fit into 1,440 minutes, + and two steps longer than a day in total overlap only by the minutes beyond a full day. Example (best-case LRV 10 = UV 4 + UF 4 + other steps 2): UV down 60 min and UF down 120 min give 4.2 % of the day at LRV 6, 8.3 % at LRV 6 and 87.5 % at LRV 10, a mixed LRV of about 6.9. UV and UF each down 900 min overlap by 360 min: 25 % at LRV 2 and 2 × 37.5 % at LRV 6, a mixed LRV of about 2.6. Worst-case loses all failing steps' - LRVs for the whole day. With three or more failing steps, the unavoidable overlap is - placed on the steps whose joint loss gives the lowest mixed concentration; in general, - best-case arranges the failure events of a day so that the mixed water has the lowest - concentration, which always implies minimal overlap (decided 2026-09-24). + LRVs for the whole day. If three or more steps fail on the same day and their + durations add up to more than 1,440 minutes, best-case counts that day like worst-case: + all these steps are down for the whole day. Only steps that lose LRV for the pathogen + group count (decided 2026-09-25, replacing the + lowest-concentration arrangement of 2026-09-24, which needed a numerical solver for a + rare exception). 7. **Results.** Only the result including failures is shown, and the reference-level exceedance is taken on it. The effect of failures is seen by comparing scenarios with different failure inputs in the assessment comparison view. That view compares annual diff --git a/docs/failure_roadmap.md b/docs/failure_roadmap.md index c6a25f7..9df4205 100644 --- a/docs/failure_roadmap.md +++ b/docs/failure_roadmap.md @@ -21,7 +21,7 @@ at least one positive LRV accept failure inputs (D4); values of a personal treat carried into the assessment; saving, reopening and editing keep them. Existing saved assessments get failure frequency 0, so their results are unchanged. -Not included: any effect on the calculation (C2), export and report (C3), explanations +Not included: any effect on the calculation (C2), the export (C3), explanations beyond field labels and units (C4). ### C2 Failure calculation — Large @@ -31,25 +31,29 @@ Spec issue: #___ (draft: `docs/specs/c2-failure-calculation.md`) Purpose: results include treatment failures as decided in ADR-0001. Scope: failure days drawn per exposure event and per treatment step inside the Monte Carlo -simulation; worst-case loses the full LRV of every failing step; best-case applies the +simulation; worst-case loses every positive LRV of every failing step; best-case applies the mixed-water assumption; a failure removes only positive LRVs, LRVs of 0 or below stay -(D4); combined failures overlap as little as possible; the result page +(D4); combined failures that fit into a day do not overlap, two steps longer than a day +overlap only by the extra minutes, and three or more steps longer than a day in total count +like worst-case; the result page and the reference-level exceedance show the result including failures. With every failure frequency at 0, results equal today's. Tested with checks that need no external expected values: failure frequency 0 reproduces today's results exactly, worst-case risk is never below best-case, a higher failure frequency never lowers the risk, LRVs of 0 or below are -unaffected, and combined failures overlap minimally in best-case. +unaffected, and combined failures follow the best-case rules for overlap. Not included: the comparison with an independent calculation (C5), showing normal operation next to the result with failures (decided against), DALYs in the assessment comparison (later). -### C3 Failure inputs in export and report — Small +### C3 Failure inputs in the export — Small -The export package's treatment table and the HTML report show each treatment step's failure -frequency and failure duration, so a reader can see which failures a result is based on. -Left out: renaming export files (`inflows.csv` → `inflow_concentration.csv` waits until -later), origin and reference columns for inputs, changes to the assessment comparison. +The export package's treatment table shows each treatment step's failure frequency and +failure duration, so a reader can see which failures a result is based on. Left out: the +HTML report, which shows no inputs at all and so doesn't show failure inputs either +(decided 2026-09-25); renaming export files (`inflows.csv` → `inflow_concentration.csv` +waits until later); origin and reference columns for inputs; changes to the assessment +comparison. ### C4 Explain treatment failure — Small diff --git a/docs/progress.md b/docs/progress.md new file mode 100644 index 0000000..a5bb425 --- /dev/null +++ b/docs/progress.md @@ -0,0 +1,101 @@ +# Progress: treatment failure (C1–C5) + +Status: 2026-09-29. Work of 2026-09-24 to 2026-09-28 on the Changes in `docs/failure_roadmap.md`. +Nothing is merged into `main` (roadmap decision D5): each Change is a draft pull request that +targets `main` only so that the pipeline deploys it to dev. The production jobs run only for +`main` itself. + +## Overview + +| Change | Branch (based on) | Pull request | Tickets | State | +|---|---|---|---|---| +| C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`) | Built, tested manually on dev | +| C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | Built, tested manually on dev | +| C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open | +| C4 Explain treatment failure | not created yet (will be based on C2) | — | #32, #33 (`12`, `13`) | Tickets ready | +| C5 Failure benchmark | — | — | — | Waits for decisions D1–D3 | + +**Dev** (https://dev.qmra.org) runs C1 + C2 + C3 (PR #34, commit `27b5268`) since 2026-09-25. + +**Review of 2026-09-28** (by a teammate): +- **Statuses:** tickets 04–11 are set to "review". +- **Review notes:** ticket 09 records a review of the C2 commits against the Spec, ADR-0001 and the code; tickets 10 and 11 record a review of the C3 commits. +- **D4 wording:** `CONTEXT.md`, the roadmap and the vision now say that worst-case loses every *positive* LRV of a failing step, as the code does. + +## What each Change delivers + +### C1 Failure inputs (#28) + +- **Configurator (#21, `57bdd21`):** + - each treatment step has a failure frequency (days per year, 0–365, default 0) and a failure duration (whole minutes, 1–1440, default 30), with units; + - values are checked on run and save, and stay with their step through save, reopen and edit; + - the migration gives existing assessments 0 and 30; + - one shared validation rule replaces the duplicated min ≤ max LRV check. +- **Positive LRV and guests (#22, `52e6087`):** + - a frequency above 0 needs at least one positive LRV (D4); + - guests now see validation messages on the affected field and step (before, an invalid guest result failed silently); + - the treatments section grows with its content instead of hiding the new fields under the footer. +- **Personal treatment steps (#23, `7f05bcd`):** they store both values under the same rules, the configurator copies them into a newly added step, and the copy can be changed without touching the personal step. + +### C2 Failure calculation (#29) + +- **Tidy-up (#24, `93c8f76`):** the simulation works per exposure event. Results are bit-identical to before (checked on 28 scenarios, 1932 values). +- **Failure days and worst-case (#25, `2f6efec`):** + - each exposure event falls on a failure day of each step with probability frequency / 365 (ADR-0001); + - the draws come from their own fixed seed, one stream per step position, shared by both cases and all pathogens; + - worst-case loses the failing steps' positive minimum LRVs for the whole day. +- **Best-case (#26, `a8a1972`):** the mixed-water assumption. For combined failures of one day, only steps that lose LRV for the pathogen group count: + - failures that fit into a day don't overlap (one step: Eq. 5); + - two steps longer than a day overlap by the extra minutes; + - three or more steps longer than a day in total count like worst-case. +- **Without failures:** results are exactly those from before C2. +- **Local timing** (drinking water, 3 pathogens, 5 bundled steps): 0.07 s without failures, 0.13–0.19 s with failures. + +### C3 Failure inputs in the export (#34) + +- **Reference export test (#30, `7085d78`, fixes `305ec0f`, `27b5268`):** implements the "Export stability" Quality goal. + - A fixed benchmark assessment is exported and compared with `qmra/risk_assessment/tests/reference_export/`: the same files, identical text, and each number within 0.1 % relative. Plots are only checked for presence. + - `UPDATE_REFERENCE_EXPORT=1` rewrites the reference on purpose (see the README there). +- **Failure columns (#31, `a04e2f9`):** `treatments.csv` has "Failure frequency (days per year)" and "Failure duration (minutes)", repeated on each step's three rows. The HTML report stays without inputs. +- **For the release notes:** `treatments.csv` has two new columns at the end. + +## Decisions made during implementation + +All on 2026-09-25 unless noted. The documents named are updated. + +- **C1: failure fields are required in the form.** Existing form tests got the two fields in their input data. +- **C1: guests see validation errors.** The guest result returns the messages, and the configurator shows them on the affected step (option a). +- **C1: dev only moves forward.** The migrations can't be reversed on dev, and only C1 and branches built on it go there until the release. The columns get no database default. +- **C2: statistics instead of the mean.** Tests check the returned statistics and the exceedance, because the mean isn't returned. Per-year coupling makes that equivalent. +- **C2: one C1 test split.** With frequency 0, results and export stay identical. With failures, only values change. +- **C2: three or more steps over a day count like worst-case in best-case.** This replaces the lowest-concentration arrangement decided on 2026-09-24, which needed a linear program whose solver failed for large combined losses; the per-pathogen arrangement rule is dropped with it. See the C2 Spec, `CONTEXT.md` (*Combined failure*), decision 6 in `docs/QMRA_Failure_Approach.md`, the roadmap and the vision. +- **C2: only steps that lose LRV for the pathogen group count** towards "three or more" (Reading 1). +- **C2: tested on dev with a PR into `main`,** because the pipeline deploys only pull requests into `main`. +- **C3 and C4 have no Spec** (Small Changes). The Tickets follow the roadmap entries. +- **C3: the HTML report shows no inputs** and doesn't get failure inputs. Roadmap entry corrected. +- **C3: failure columns are repeated on each step's three rows** of `treatments.csv`. +- **C3: a reference export test is added**, comparing numbers within 0.1 %. CI's processor and math library differ from this server's by up to 10⁻⁵ relative in tiny best-case probabilities. +- **C4: texts are drafted by Claude** and reviewed by a domain expert before the release. Step 4 of the guided tour is extended, and a new Sphinx page is added. +- **C3 and C4 are based on C2,** so dev keeps C1 + C2 while they are tested. + +## Open items + +**Checks still to do** +- **C1 (#28):** real-data check; "I can explain this Change". +- **C2 (#29):** calculation time measured on dev and recorded in the PR (ticket 09, #26); real-data check; "I can explain this Change". +- **C3 (#34):** manual check on dev, from the list in the PR; real-data check; "I can explain this Change". + +**Work not started** +- **C4:** #32 (FAQ and guided tour) and #33 (Sphinx page) can start now. +- **C5:** needs decisions D1 (tolerance), D2 (who writes the independent script) and D3 (who signs off). The script must implement the combined-failure rule of 2026-09-25. + +**Known issues and risks** +- **Bug #27** (older than C1): an empty "Events per year" or "Volume per event" gives a server error for guests and registered users. +- **Lost click** (existing behaviour): typing into a field and clicking "Show assessment results" straight away loses the first click. +- **Precision:** tiny best-case probabilities lose precision in `1 − exp(−k·dose)`. Making the calculation more precise (`expm1`/`log1p`) would be a separate change to the calculation. +- **Step order:** failure draws are keyed by a step's position in the train. Saved assessments pass their steps without an explicit order (stable in practice). +- **Unpinned dependencies:** numpy and pandas aren't pinned; reproducibility across releases is an open question for Architecture. +- **Housekeeping:** + - the C1 and C2 Spec issues were never created (`#___` in the roadmap); + - the local branch `c2-failure-calculation` on the Controlled server is behind GitHub (the review commits of 2026-09-28) and holds one ticket commit (`5ca99b1`) that is on GitHub only through `c3-failure-inputs-in-export`. Update it from GitHub before working on it again. + - The GitHub command-line login on the Controlled server has expired (`HTTP 401`); issues and pull requests can't be updated from there until someone logs in again. diff --git a/docs/specs/c2-failure-calculation.md b/docs/specs/c2-failure-calculation.md index 093203e..a484904 100644 --- a/docs/specs/c2-failure-calculation.md +++ b/docs/specs/c2-failure-calculation.md @@ -24,8 +24,10 @@ treatment step with probability failure frequency / 365. On a failure day: whole day, whatever the failure duration; - **best-case** applies the mixed-water assumption: the consumed water is a mixture of water treated during the failure events and water treated in normal operation, in proportion to - the failure durations, with the failure events arranged within the day so that the mixed - water has the lowest pathogen concentration. + the failure durations. Failure events that fit into one day do not overlap; two steps + longer than a day in total overlap only by the minutes beyond the day; three or more steps + longer than a day in total count like worst-case for that day. Only steps that lose LRV + for the pathogen group count. LRVs of 0 or below are never affected (D4). The result page, the reference-level exceedance, the saved results and the result table in the export package all show the @@ -60,9 +62,11 @@ failure frequency is 0 gives exactly the same results as before C2. day to be combined, so that combined failures are not missed. 12. As a user, I want a combined failure in best-case to assume that failure events do not overlap when they fit into one day, so that best-case stays the favourable assumption. -13. As a user, I want combined failure events longer than a day in total to overlap only as - much as necessary, on the steps whose joint loss matters least, so that best-case stays - favourable without a jump at 24 hours. +13. As a user, I want two failure events longer than a day in total to overlap only by the + minutes beyond the day, so that best-case stays favourable without a jump at 24 hours. + When three or more steps fail on the same day for longer than a day in total, I accept + that best-case counts that day like worst-case, since so much downtime on one day is a + rare exception. 14. As a user, I want a failure to remove only the positive LRVs of a step, so that a step with regrowth (a negative LRV) in one pathogen group does not improve during a failure. 15. As a user, I want a pathogen group whose LRV at a step is 0 or empty to be unaffected by @@ -113,13 +117,19 @@ failure frequency is 0 gives exactly the same results as before C2. - **Best-case on a failure day (mixed-water assumption).** The day is split into time segments by which failing steps are down. The event's LRV is −log₁₀ of the time-weighted average of 10^(−LRV of each segment), where a segment's LRV is the best-case LRV of the - train minus the removable maximum LRVs of the steps down in it. The failure events of the - day are arranged within 1,440 minutes so that this average concentration is lowest. This - gives no overlap when the durations fit into one day and otherwise the minimal overlap, - placed on the steps whose joint loss matters least (`CONTEXT.md`, *Combined failure*). For - a single failing step it equals Eq. 5 of the failure approach. With the few failing steps - per day that occur in practice, the arrangement can be found exactly, for example by - choosing the time share of each combination of failing steps. + train minus the removable maximum LRVs of the steps down in it (`CONTEXT.md`, *Combined + failure*): + - failure events whose durations fit into 1,440 minutes do not overlap; for a single + failing step this equals Eq. 5 of the failure approach; + - two failure events longer than a day in total overlap only by the minutes beyond the day; + - three or more failure events longer than a day in total count like worst-case: all these + steps are down for the whole day and lose their removable maximum LRVs. Only steps with a + removable maximum LRV for the pathogen group count, since a failure changes nothing where + a step removes nothing (D4); so a day can count like worst-case for one reference + pathogen and not for another. This replaces + the lowest-concentration arrangement decided on 2026-09-24, which needed a numerical + solver for a rare exception (decided 2026-09-25). + Every rule has a closed formula, so no solver is needed. - **Concentration sampling unchanged.** The sampled inflow concentrations and the choice of which sample each simulated event uses stay exactly as today, including the fixed seed. The failure draws come from their own fixed-seed random generator. When all failure @@ -162,8 +172,10 @@ failure frequency is 0 gives exactly the same results as before C2. give the same best-case results as the scenario with the train's best-case LRV replaced by the constant mixed LRV from Eq. 5, computed by hand in the test. - **Combined failures:** two steps always failing with 900 + 900 minutes equal the - hand-computed constant LRV with 360 minutes of overlap; three steps always failing with - durations above a day in total equal the hand-computed lowest-concentration arrangement. + hand-computed constant LRV with 360 minutes of overlap; three steps always failing within + a day in total equal the hand-computed LRV without overlap; three steps always failing + with more than a day in total equal the scenario without their positive maximum LRVs; + a failing step without removal for one pathogen group does not count for that group. - **Monotonicity:** raising a failure frequency never lowers any mean; raising a failure duration never lowers a best-case mean and leaves worst-case unchanged. - **Case order:** worst-case mean ≥ best-case mean for every pathogen and risk measure, @@ -183,8 +195,8 @@ failure frequency is 0 gives exactly the same results as before C2. ## Acceptance criteria 1. The calculation follows the rules above: failure days per exposure event, worst-case - full loss, best-case mixed-water assumption with the lowest-concentration arrangement, - and only positive LRVs lost. + full loss, best-case mixed-water assumption with the combined-failure rules above, and + only positive LRVs lost. 2. With every failure frequency at 0, all results equal today's exactly. 3. All tests listed above pass, and the existing suite passes unchanged. 4. The result page, the reference-level exceedance, saved results, the assessment comparison @@ -220,11 +232,12 @@ data* (no change to access). ## Further Notes -- The rule for arranging three or more failure events of one day in best-case was decided - on 2026-09-24 while writing this Spec and added to `CONTEXT.md` (*Combined failure*) and - to decision 6 in `docs/QMRA_Failure_Approach.md`. Overlapping two failure events never - lowers the mixed concentration, so the lowest-concentration arrangement always has - minimal overlap and agrees with the rule for two steps. +- The rule for three or more failure events of one day in best-case changed during + implementation (#26, 2026-09-25). The lowest-concentration arrangement decided on + 2026-09-24 needed a linear program whose standard solver failed when the failing steps + lose about 16 LRV or more together. Since such days are a rare exception, they now count + like worst-case. `CONTEXT.md` (*Combined failure*) and decision 6 in + `docs/QMRA_Failure_Approach.md` are updated. - No contradiction with ADR-0001 was found. ADR-0001 notes that two exposure events on the same calendar day need not share a failure state; this Spec keeps that. - Results with failures will differ from a hand calculation with Eq. 6 of the failure diff --git a/docs/tickets/04-c1-failure-inputs-in-configurator.md b/docs/tickets/04-c1-failure-inputs-in-configurator.md new file mode 100644 index 0000000..eaa2ee0 --- /dev/null +++ b/docs/tickets/04-c1-failure-inputs-in-configurator.md @@ -0,0 +1,24 @@ +# 04: Failure inputs on a treatment step in the configurator: enter, check ranges, save, reopen, edit + +**What to build:** Every treatment step in the assessment configurator gets two more fields: **failure frequency** (failure days per year) and **failure duration** (minutes). Registered users can save an assessment with these values, and when they reopen or edit it they see the same values again. The values do not change the result yet; the calculation follows in C2. + +Tidy-up comes first. The configurator's treatment step form and the personal treatment step form both contain the same minimum ≤ maximum LRV check. It becomes one shared rule that both forms use, and the failure rules are added to it later. This Ticket also adds the first HTTP-level tests. They use Django's test client and a logged-in test user, and they check what a browser or a later request can see. + +Delivered on branch `c1-failure-inputs` as part of the single C1 pull request. The pull request stays open until the release (D5). + +Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`), Change C1 in `docs/failure_roadmap.md`. The Spec issue does not exist yet. GitHub issue #21. + +**Blocked by:** None (can start immediately) + +**Status:** review + +- [x] The minimum ≤ maximum LRV check is defined once and used by both treatment step forms. Nothing changes that a user can notice, and the existing test suite passes unchanged. +- [x] A schema migration adds failure frequency (a decimal number) and failure duration (a whole number) to the treatment steps of an assessment. Existing rows get failure frequency 0 and failure duration 30. +- [x] Each treatment step card in the configurator shows both fields below its LRV rows, labelled "Failure frequency (days per year)" and "Failure duration (minutes)". New steps start at 0 and 30, bundled steps included. +- [x] Failure frequency outside 0–365, and failure duration outside 1–1440 or not a whole number, are rejected with a message on the saved configurator. The duration range is checked even when the frequency is 0. +- [x] A fractional failure frequency such as 0.5 is accepted. +- [x] Saving an assessment and reopening it gives back the same failure values for each step. Editing them and saving again stores the new values. +- [x] Each value stays with its own step when other steps are removed. Two copies of the same bundled step can have different values. +- [x] An assessment created without failure values, as before the migration, reopens with failure frequency 0 and failure duration 30. +- [x] An assessment's result and export package are the same with and without failure inputs. +- [x] All of the above is covered by HTTP-level tests through Django's test client, and the existing test suite passes unchanged. diff --git a/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md b/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md new file mode 100644 index 0000000..ddcc45f --- /dev/null +++ b/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md @@ -0,0 +1,20 @@ +# 05: Failure frequency only on steps with a positive LRV (D4), with messages for guests too + +**What to build:** A treatment step can only have a failure frequency above 0 if at least one of its LRVs is positive, because a failure removes only positive LRVs (D4). The configurator checks this when the assessment is run or saved, and the error appears on the failure frequency field. + +Guests can use the failure fields too, and the same rules apply to them. Today, when a guest runs an assessment with invalid input, the guest result sends back an empty error and nothing appears on the page. With this Ticket, the guest result sends back the error messages and the configurator shows them on the affected treatment step, as it does for registered users. This also makes the existing LRV errors, such as minimum above maximum, visible to guests. + +Delivered on branch `c1-failure-inputs` as part of the single C1 pull request. + +Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`), roadmap decision D4. The Spec issue does not exist yet. GitHub issue #22. + +**Blocked by:** 04 (#21) + +**Status:** review + +- [x] A failure frequency above 0 is rejected, with a message on the failure frequency field, if none of the step's six LRVs (minimum and maximum for bacteria, viruses and protozoa) is above 0. An empty LRV counts as 0. +- [x] A step whose only positive LRV is one maximum (for example protozoa 0–2) accepts a failure frequency above 0. +- [x] A step with only negative LRVs (recontamination) or only empty LRVs keeps failure frequency 0. A higher value is rejected. +- [x] A guest sees the failure fields, can run an assessment with valid failure inputs, and nothing is stored. +- [x] When a guest's input is invalid, whether a failure input or an LRV, the guest result sends back the error messages and the configurator shows them on the affected step. The rules are the same as for registered users. +- [x] HTTP-level tests cover the D4 cases on the saved configurator and on the guest result, and cover rejection with a message on the guest result. The existing test suite passes unchanged. diff --git a/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md b/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md new file mode 100644 index 0000000..a369fb2 --- /dev/null +++ b/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md @@ -0,0 +1,24 @@ +# 06: Personal treatment steps store failure inputs and the configurator copies them in + +**What to build:** A registered user can store a failure frequency and a failure duration on a personal treatment step, with the same rules as in the configurator. When they add the step to an assessment, its failure values are copied in together with its LRVs. The copy can then be changed in that one assessment without changing the personal treatment step. + +Today the personal treatment step form sends back an empty error for invalid input, so the user sees nothing. With this Ticket it shows the messages, the same way it already does for a duplicate name. + +Delivered on branch `c1-failure-inputs` and closes the C1 pull request. The pull request is tested on the dev environment and not merged before the release (D5). + +Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`). The Spec issue does not exist yet. GitHub issue #23. + +**Blocked by:** 04 (#21), 05 (#22) + +**Status:** review + +- [x] A schema migration adds failure frequency and failure duration to personal treatment steps. Existing personal treatment steps get failure frequency 0 and failure duration 30. +- [x] The personal treatment step form shows both fields with their units and applies the same shared rule as the configurator: the ranges, whole minutes, D4 and minimum ≤ maximum LRV. Invalid input is rejected and the message is shown in the form. +- [x] A personal treatment step saved with failure inputs appears with them in its owner's list of personal treatment steps. Another user's list does not contain it. +- [x] Adding a personal treatment step to an assessment copies its failure frequency and failure duration into the new step. Bundled steps still start at 0 and 30. +- [x] Changing the copied values in an assessment and saving leaves the personal treatment step unchanged. +- [x] HTTP-level tests cover the form rules, old personal treatment steps reading back 0 and 30, and owner-only visibility. The existing test suite passes unchanged. +- [ ] Manual check on the dev environment, announced to the team first: + - Create a personal treatment step with failure inputs and add it to an assessment. + - See the copied values, change them, save and reopen. + - Confirm that the personal treatment step itself is unchanged. diff --git a/docs/tickets/07-c2-calculation-per-exposure-event.md b/docs/tickets/07-c2-calculation-per-exposure-event.md new file mode 100644 index 0000000..52c966f --- /dev/null +++ b/docs/tickets/07-c2-calculation-per-exposure-event.md @@ -0,0 +1,18 @@ +# 07: Tidy-up: the calculation works per exposure event, with results unchanged + +**What to build:** Tidy-up before failures enter the calculation. No result changes. + +The Monte Carlo simulation is reorganised to work per simulated exposure event. Each event picks its concentration sample by position instead of by value, which uses exactly the same random numbers as today, and each event gets its own LRV. For now every event still gets the same LRV. Best-case and worst-case use the same building blocks, and the summary statistics (mean, minimum, quartiles, median, maximum) and the reference-level exceedance are computed in one place for both cases and both risk measures. The calculation's public entry point keeps its interface. + +Delivered on branch `c2-failure-calculation`, based on `c1-failure-inputs`, as part of the single C2 pull request. The pull request stays open until the release (D5). + +Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), Change C2 in `docs/failure_roadmap.md`. The Spec issue does not exist yet. GitHub issue #24. + +**Blocked by:** None (can start immediately) + +**Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) + +- [x] Every simulated exposure event has its own concentration sample, picked by position, and its own LRV. The random numbers are the same as today's. +- [x] Best-case and worst-case share the same calculation steps. Summary statistics and the reference-level exceedance are computed in one place. +- [x] The existing regression test passes with exactly the same numbers, and the existing test suite passes unchanged. +- [x] A new test at the calculation's public entry point shows that calculating the same scenario twice gives identical results. diff --git a/docs/tickets/08-c2-failure-days-and-worst-case.md b/docs/tickets/08-c2-failure-days-and-worst-case.md new file mode 100644 index 0000000..070879b --- /dev/null +++ b/docs/tickets/08-c2-failure-days-and-worst-case.md @@ -0,0 +1,21 @@ +# 08: Failure days drawn per exposure event; worst-case loses the positive LRVs of failing steps + +**What to build:** The worst-case result includes treatment failures. For every simulated exposure event, and for every treatment step with a failure frequency above 0, the step fails that day with probability failure frequency / 365 (ADR-0001). On a failure day, worst-case calculates the event with the positive minimum LRVs of every failing step removed, whatever the failure duration. LRVs of 0, below 0 or empty stay in place (D4). + +The failure draws come from their own fixed-seed random generator and are shared by both cases and all reference pathogens. Raising a failure frequency therefore only adds failure days. If every failure frequency is 0, no failure draws are made and the results are exactly today's. Best-case does not change in this Ticket; the mixed-water assumption follows in the next one. + +The result page, the reference-level exceedance, saved results and the export's result table show whatever the calculation returns. Their layout does not change. + +Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), ADR-0001, roadmap decision D4. The Spec issue does not exist yet. GitHub issue #25. + +**Blocked by:** 07 (#24), 04 (#21, C1 stored failure inputs) + +**Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) + +- [x] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. +- [x] A step with failure frequency 365 gives the same worst-case results as the same scenario without that step's positive minimum LRVs. +- [x] Raising a failure frequency never lowers any mean. Changing a failure duration leaves worst-case unchanged. +- [x] Worst-case mean ≥ best-case mean for every reference pathogen and risk measure, with and without failures. +- [x] D4: a failing step with a negative LRV in one pathogen group and a positive LRV in another leaves the first group's results unchanged. A group with LRV 0 is unaffected. +- [x] Calculating the same scenario with failures twice gives identical results. +- [x] All tests go through the calculation's public entry point and check only the returned results. The existing test suite passes unchanged. diff --git a/docs/tickets/09-c2-best-case-mixed-water.md b/docs/tickets/09-c2-best-case-mixed-water.md new file mode 100644 index 0000000..5211614 --- /dev/null +++ b/docs/tickets/09-c2-best-case-mixed-water.md @@ -0,0 +1,40 @@ +# 09: Best-case applies the mixed-water assumption, including combined failures + +**What to build:** The best-case result includes treatment failures under the mixed-water assumption. On a failure day, the consumed water is a mixture of water treated during the failure events and water treated in normal operation, in proportion to the failure durations. The event's LRV is −log₁₀ of the time-weighted average of 10^(−LRV) over the segments of the day, where a segment's LRV is the best-case LRV of the train minus the positive maximum LRVs of the steps down in that segment. + +Combined failures of one day: +- if their durations fit into one day, they do not overlap; +- two failure events longer than a day in total overlap only by the minutes beyond the day; +- three or more failure events longer than a day in total count like worst-case: all these steps are down for the whole day (changed on 2026-09-25 from the lowest-concentration arrangement, which needed a numerical solver for a rare exception). Only steps that lose LRV for the pathogen group count. + +For a single failing step, the result equals Eq. 5 of the failure approach. Work per failure day is reused for repeated combinations of failing steps. + +This Ticket closes C2: it includes the manual check on the dev environment and the time measurement. + +Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), ADR-0001, `CONTEXT.md` (*Mixed-water assumption*, *Combined failure*), decision 6 in `docs/QMRA_Failure_Approach.md`. The Spec issue does not exist yet. GitHub issue #26. + +**Blocked by:** 08 (#25) + +**Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) + +- [x] A step with failure frequency 365 and failure duration 1,440 gives the same best-case results as the same scenario without that step's positive maximum LRVs. +- [x] A step with failure frequency 365 and failure duration 60 gives the same best-case results as the scenario with the train's best-case LRV replaced by the Eq. 5 mixed LRV, computed by hand in the test. +- [x] Two steps always failing with 900 + 900 minutes equal the hand-computed result with 360 minutes of overlap. +- [x] Three steps always failing within a day in total equal the hand-computed result without overlap. +- [x] Three steps always failing with more than a day in total equal the scenario without their positive maximum LRVs. +- [x] A failing step without removal for one pathogen group does not count for that group. +- [x] Raising a failure duration never lowers a best-case mean, and raising a failure frequency never lowers any mean. +- [x] Worst-case mean ≥ best-case mean for every reference pathogen and risk measure, with failures. +- [x] With every failure frequency at 0, all results equal today's exactly. The same scenario gives identical results every time. The existing test suite passes unchanged. +- [x] Manual check on the dev environment, announced to the team first: + - Enter a scenario with failures and run it as a guest and as a registered user. + - Confirm that the result page, the saved-assessments page, the assessment comparison and the export's result table show the changed results, with unchanged layout. +- [ ] The calculation time for a drinking-water scenario (365 events per year) with failures is measured on the dev environment and recorded in the pull request. + +Review on 2026-09-28 (against the Spec, ADR-0001 and the code of the three C2 commits): + +- [x] The calculation matches the Spec: failure days per exposure event from a fixed-seed generator shared by both cases and all pathogens; worst-case loses only positive minimum LRVs; best-case follows the mixed-water rules including the three-or-more exception; LRVs of 0 or below are untouched. The hand-computed examples of decision 6 (mixed LRV about 6.9 and about 2.6) agree with the code's formula. +- [x] The concentration samples are picked by position with the same generator calls as before, so results without failures stay bit-identical. The PR records a 28-scenario check of this. +- [x] Wording left over from before D4 (fixed 2026-09-28): `CONTEXT.md` (*Combined failure*: "loses all their LRVs"), `docs/vision.md` MVP item 3 and the C2 scope in `docs/failure_roadmap.md` still say worst-case loses the *full* LRV of a failing step. It should say every *positive* LRV. +- [ ] Decide whether it is acceptable that the failure draws of a step depend on its position in the train. Two scenarios that differ only by an extra non-failing step placed *before* a failing one give slightly different results, because the later step's random stream changes. The difference is sampling noise, but it can surprise a user comparing two saved assessments, and C5's independent script must use the same keying to match. +- [ ] The Spec still says "Status: draft" and "Spec issue: #___". Set it to implemented and create the Spec issue, or record that C2 has none. diff --git a/docs/tickets/10-c3-reference-export-test.md b/docs/tickets/10-c3-reference-export-test.md new file mode 100644 index 0000000..941626f --- /dev/null +++ b/docs/tickets/10-c3-reference-export-test.md @@ -0,0 +1,29 @@ +# 10: Reference test for the export package + +**What to build:** Tidy-up before the export package changes. The "Export stability" Quality goal asks for a test that compares the export package of a benchmark assessment with a stored reference export, so that the content and layout of the export package change only on purpose. That test doesn't exist yet. + +A fixed benchmark assessment is exported through the application and compared with a reference export stored in the repository: +- the same files in the ZIP; +- every CSV table and the HTML report identical; +- the two plots present. Their image bytes aren't compared, because plot rendering can differ between machines. + +A short, documented way to update the reference on purpose is part of this Ticket. The export itself doesn't change. + +Delivered on a new branch based on `c2-failure-calculation`, together with #31 as the C3 pull request. The PR targets `main` as a draft only so that it deploys to dev, and it is not merged before the release (D5). + +Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), the "Export stability" Quality goal in `docs/vision.md`, and the decisions of 2026-09-25. GitHub issue #30. + +**Blocked by:** None (can start immediately) + +**Status:** review (built in pull request #34, draft; CI green on `27b5268`; stays open until the release, D5) + +- [x] A test exports a fixed benchmark assessment and compares it with the stored reference export: the same files, identical CSV tables and HTML report, and both plots present. +- [x] The test fails when a column, a file or a value of the export changes, and its message says which file differs. +- [x] How to update the reference on purpose is documented next to the reference files. +- [x] The export package itself is unchanged, and the existing test suite passes. + +Review on 2026-09-28 (commits `7085d78`, `305ec0f`, `27b5268`): + +- [x] The test exports a fixed benchmark assessment with fixed inputs, not bundled data, and one failing step, so it also covers the C2 calculation. Plots are checked for presence only and masked inside the report. +- [x] The comparison is not byte-identical, as this ticket first said: text must be identical and each number within 0.1 % relative. The reason is recorded in the README and the PR: tiny probabilities from 1 − exp(−k · dose) differ by up to 10⁻⁵ between CI and the server. A renamed column, a missing file or a 1 % change still fails. +- [ ] Follow-up, not part of C3: computing those probabilities with `expm1` would remove the cancellation and allow a tighter tolerance. That is a change to the calculation and needs its own ticket if wanted. diff --git a/docs/tickets/11-c3-failure-inputs-in-export.md b/docs/tickets/11-c3-failure-inputs-in-export.md new file mode 100644 index 0000000..cca3996 --- /dev/null +++ b/docs/tickets/11-c3-failure-inputs-in-export.md @@ -0,0 +1,26 @@ +# 11: Failure inputs in the treatment table of the export + +**What to build:** The export package records which failures a result is based on. The treatment table `treatments.csv` gets two more columns, "Failure frequency (days per year)" and "Failure duration (minutes)". They are repeated on each of the step's three rows (one per pathogen group), so the file keeps its layout of one row per step and pathogen group. + +The HTML report stays as it is: it shows no inputs at all today, so it doesn't show failure inputs either. The roadmap's C3 entry is corrected to say so. + +This is an intended change to the export package, so the reference export from #30 is updated in the same commit. The pull request notes the change for the release notes (the repository has no release-notes file). + +Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), decisions of 2026-09-25 (report without inputs; failure columns repeated on the step's rows). GitHub issue #31. + +**Blocked by:** 10 (#30) + +**Status:** review (built in pull request #34, draft; CI green on `27b5268`; stays open until the release, D5) + +- [x] `treatments.csv` has the columns "Failure frequency (days per year)" and "Failure duration (minutes)", with each step's values on all three of its rows. +- [x] An assessment saved with failure inputs exports exactly those values per step, and an assessment without failure inputs exports 0 and 30. +- [x] No other file of the export package changes; the HTML report shows no inputs, as before. +- [x] The reference export is updated in the same commit, and the reference test passes. +- [x] The roadmap's C3 entry says that the HTML report is left out because it shows no inputs. +- [x] The pull request notes the export change for the release notes. + +Review on 2026-09-28 (commit `a04e2f9`): + +- [x] Only `treatments.csv` changed in the reference export, in the same commit as the code; later commits touched only the test tolerance and the README. +- [x] Assessments saved before C1 export 0 and 30, because the new fields have those defaults. +- [ ] Manual check on dev from PR #34 still open: download an export with failure inputs, check the two columns and the unchanged report. diff --git a/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md b/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md new file mode 100644 index 0000000..43bf0b4 --- /dev/null +++ b/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md @@ -0,0 +1,30 @@ +# 12: Explain treatment failure in the FAQ and the guided tour + +**What to build:** Users of the application learn what the failure inputs mean and how the result uses them, without reading the method documents. + +- **FAQ:** a new question on treatment failure explains: + - failure frequency (failure days per year) and failure duration (minutes); + - that failure days are drawn at random per exposure event (ADR-0001); + - that a failure removes only the positive LRVs of a step (D4); + - why worst-case ignores the failure duration; + - the mixed-water assumption in best-case; + - combined failures: no overlap within a day, two steps overlapping by the minutes beyond a day, and three or more steps longer than a day in total counting like worst-case, counting only steps that lose LRV for the pathogen group. + + The limitations question gets one line: partial loss of removal isn't modelled. +- **Guided tour:** step 4 (Treatment) gets one or two sentences on the two failure fields. + +The text uses the glossary terms from `CONTEXT.md`. It is a draft written by Claude and must be reviewed by a domain expert before the release. The pull request says so. + +Delivered on a new branch based on `c2-failure-calculation`, together with #33 as the C4 pull request (a draft into `main` only so that it deploys to dev; not merged before the release, D5). + +Source: Change C4 in `docs/failure_roadmap.md` (Small, no Spec), `CONTEXT.md` (Treatment failure, Failure day, Failure frequency, Failure duration, Mixed-water assumption, Combined failure), ADR-0001, decisions 1–8 in `docs/QMRA_Failure_Approach.md`, the C2 Spec, and the decisions of 2026-09-25. GitHub issue #32. + +**Blocked by:** None (can start immediately) + +**Status:** ready-for-agent + +- [ ] The FAQ has a question on treatment failure covering all points above, consistent with `CONTEXT.md` and the implemented calculation. +- [ ] The FAQ's limitations mention that partial loss of removal isn't modelled. +- [ ] Step 4 of the configurator's guided tour mentions failure frequency and failure duration with their units. +- [ ] Deferred UI wording (such as "tolerable risk level") and explanations of unrelated parts of the model are left as they are. +- [ ] The pull request marks the text as a draft for domain expert review. diff --git a/docs/tickets/13-c4-sphinx-page-treatment-failure.md b/docs/tickets/13-c4-sphinx-page-treatment-failure.md new file mode 100644 index 0000000..89696fb --- /dev/null +++ b/docs/tickets/13-c4-sphinx-page-treatment-failure.md @@ -0,0 +1,24 @@ +# 13: Sphinx page Treatment failure + +**What to build:** The Sphinx documentation (`docs/source/`) gets a page "Treatment failure", linked from `index.rst`. It explains the method in more detail than the FAQ: +- the inputs and their ranges; +- failure days per exposure event, and why the application departs from the 365-day sequence of the source method (ADR-0001); +- worst-case full loss for the whole day; +- the mixed-water assumption with Eq. 5; +- the rules for combined failures; +- D4 (only positive LRVs are lost). + +It cites ADR-0001 and `docs/QMRA_Failure_Approach.md`. + +The text is a draft written by Claude for domain expert review before the release. + +Source: Change C4 in `docs/failure_roadmap.md` (Small, no Spec), the same method sources as #32. GitHub issue #33. + +**Blocked by:** None (can start immediately) + +**Status:** ready-for-agent + +- [ ] `docs/source/` has a page "Treatment failure" covering the points above, linked from `index.rst`. +- [ ] The documentation builds without errors, checked in a local Sphinx environment (Sphinx isn't in the requirements). +- [ ] The page uses the glossary terms and agrees with `CONTEXT.md` and the implemented calculation. +- [ ] The pull request marks the text as a draft for domain expert review. diff --git a/docs/vision.md b/docs/vision.md index 84a6b3c..f3f3d5f 100644 --- a/docs/vision.md +++ b/docs/vision.md @@ -77,7 +77,7 @@ Functionalities, each one Change: 1. **Failure inputs per treatment step.** In the assessment configurator, each treatment step with at least one positive LRV gets a failure frequency (failure days per year, 0–365, default 0) and a failure duration (minutes, 1–1440, default 30). The inputs belong to the scenario. 2. **Failure inputs on personal treatment steps.** A personal treatment step can store its failure frequency and failure duration for reuse. -3. **Calculation with failure days.** Inside the Monte Carlo simulation, each exposure event falls on a failure day of each step with probability failure frequency / 365. Worst-case loses the full LRV of every failing step; best-case applies the mixed-water assumption, with combined failures overlapping as little as possible. +3. **Calculation with failure days.** Inside the Monte Carlo simulation, each exposure event falls on a failure day of each step with probability failure frequency / 365. Worst-case loses every positive LRV of every failing step; best-case applies the mixed-water assumption: combined failures that fit into a day do not overlap, two steps longer than a day overlap only by the extra minutes, and three or more steps longer than a day in total count like worst-case. 4. **Results and export including failures.** The result page, the reference-level exceedance and the export package show the result including failures, and the export records the failure inputs of each step. Waits until later: diff --git a/qmra/risk_assessment/exports.py b/qmra/risk_assessment/exports.py index f70aa62..f240e9c 100644 --- a/qmra/risk_assessment/exports.py +++ b/qmra/risk_assessment/exports.py @@ -27,7 +27,10 @@ def treatments_as_df(treatments: QuerySet[Treatment]) -> pd.DataFrame: "Treatment": [t.name] * 3, "Pathogen group": ["Viruses", "Bacteria", "Protozoa"], "Maximum LRV": [t.viruses_max, t.bacteria_max, t.protozoa_max], - "Minimum LRV": [t.viruses_min, t.bacteria_min, t.protozoa_min] + "Minimum LRV": [t.viruses_min, t.bacteria_min, t.protozoa_min], + # a step's failure inputs apply to all its pathogen groups + "Failure frequency (days per year)": [t.failure_frequency] * 3, + "Failure duration (minutes)": [t.failure_duration] * 3, })] return pd.concat(dfs) diff --git a/qmra/risk_assessment/forms.py b/qmra/risk_assessment/forms.py index 57b1bc7..d08f4df 100644 --- a/qmra/risk_assessment/forms.py +++ b/qmra/risk_assessment/forms.py @@ -14,6 +14,37 @@ def _zero_if_none(x): return x if x is not None else 0 +LRV_FIELDS = [ + ("bacteria_min", "bacteria_max"), + ("viruses_min", "viruses_max"), + ("protozoa_min", "protozoa_max"), +] +FAILURE_INPUT_RANGES = { + "failure_frequency": (0, 365, "days per year"), + "failure_duration": (1, 1440, "minutes"), +} + + +def check_treatment_step(form, cleaned_data): + """Minimum LRV at most maximum LRV per pathogen group, failure inputs within their ranges, + and a failure frequency above 0 only for a step with a positive LRV. + + Shared by the configurator and the personal treatment step form. Whole minutes are enforced + by the failure duration's field type; failure inputs a form does not have are skipped. + """ + for min_field, max_field in LRV_FIELDS: + if _zero_if_none(cleaned_data.get(min_field, 0)) > _zero_if_none(cleaned_data.get(max_field, 0)): + form.add_error(min_field, "min. must be less than max") + for field, (low, high, unit) in FAILURE_INPUT_RANGES.items(): + value = cleaned_data.get(field) + if value is not None and not low <= value <= high: + form.add_error(field, f"{field.replace('_', ' ')} must be between {low} and {high} {unit}") + # a failure removes only positive LRVs, so a step without one cannot fail (D4) + has_positive_lrv = any(_zero_if_none(cleaned_data.get(field)) > 0 for pair in LRV_FIELDS for field in pair) + if _zero_if_none(cleaned_data.get("failure_frequency")) > 0 and not has_positive_lrv: + form.add_error("failure_frequency", "failure frequency must be 0 for a treatment step without a positive LRV") + + class RiskAssessmentForm(forms.ModelForm): source_name = forms.ChoiceField() exposure_name = forms.ChoiceField() @@ -150,7 +181,9 @@ class Meta: 'viruses_min', 'viruses_max', "protozoa_min", - "protozoa_max" + "protozoa_max", + "failure_frequency", + "failure_duration", ] def __init__(self, *args, **kwargs): @@ -167,6 +200,10 @@ def __init__(self, *args, **kwargs): self.fields['viruses_max'].label = "" self.fields['protozoa_min'].label = "" self.fields['protozoa_max'].label = "" + self.fields['failure_frequency'].label = "Failure frequency (days per year)" + self.fields['failure_duration'].label = "Failure duration (minutes)" + # leave whole minutes to the server check: the browser blocks hidden step cards without a message + self.fields['failure_duration'].widget.attrs['step'] = "any" label_style = "class='text-muted text-center w-100' style='margin-top: .4em;'" self.helper.layout = Layout( Field("name", css_class="disabled-input d-none"), @@ -179,24 +216,13 @@ def __init__(self, *args, **kwargs): Column("viruses_min"), Column("viruses_max")), Row(Column(HTML(f"")), Column("protozoa_min"), Column("protozoa_max")), + Row(Column("failure_frequency"), Column("failure_duration")), # Row(Column("DELETE")) ) def clean(self): cleaned_data = super().clean() - b_min = _zero_if_none(cleaned_data.get("bacteria_min", 0)) - b_max = _zero_if_none(cleaned_data.get("bacteria_max", 0)) - v_min = _zero_if_none(cleaned_data.get("viruses_min", 0)) - v_max = _zero_if_none(cleaned_data.get("viruses_max", 0)) - p_min = _zero_if_none(cleaned_data.get("protozoa_min", 0)) - p_max = _zero_if_none(cleaned_data.get("protozoa_max", 0)) - msg = "min. must be less than max" - if b_min > b_max: - self.add_error("bacteria_min", msg) - if v_min > v_max: - self.add_error("viruses_min", msg) - if p_min > p_max: - self.add_error("protozoa_min", msg) + check_treatment_step(self, cleaned_data) return cleaned_data diff --git a/qmra/risk_assessment/migrations/0011_treatment_failure_inputs.py b/qmra/risk_assessment/migrations/0011_treatment_failure_inputs.py new file mode 100644 index 0000000..b05f235 --- /dev/null +++ b/qmra/risk_assessment/migrations/0011_treatment_failure_inputs.py @@ -0,0 +1,23 @@ +# Generated by Django 5.0.14 on 2026-09-25 10:10 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('risk_assessment', '0010_remove_qmratreatment_bacteria_reference_and_more'), + ] + + operations = [ + migrations.AddField( + model_name='treatment', + name='failure_duration', + field=models.IntegerField(default=30), + ), + migrations.AddField( + model_name='treatment', + name='failure_frequency', + field=models.FloatField(default=0), + ), + ] diff --git a/qmra/risk_assessment/migrations/0012_usertreatment_failure_inputs.py b/qmra/risk_assessment/migrations/0012_usertreatment_failure_inputs.py new file mode 100644 index 0000000..56c0ebd --- /dev/null +++ b/qmra/risk_assessment/migrations/0012_usertreatment_failure_inputs.py @@ -0,0 +1,23 @@ +# Generated by Django 5.0.14 on 2026-09-25 10:34 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('risk_assessment', '0011_treatment_failure_inputs'), + ] + + operations = [ + migrations.AddField( + model_name='usertreatment', + name='failure_duration', + field=models.IntegerField(default=30), + ), + migrations.AddField( + model_name='usertreatment', + name='failure_frequency', + field=models.FloatField(default=0), + ), + ] diff --git a/qmra/risk_assessment/models.py b/qmra/risk_assessment/models.py index 030def7..eaeb936 100644 --- a/qmra/risk_assessment/models.py +++ b/qmra/risk_assessment/models.py @@ -35,6 +35,8 @@ class Treatment(models.Model): viruses_max = models.FloatField(blank=True, null=True) protozoa_min = models.FloatField(blank=True, null=True) protozoa_max = models.FloatField(blank=True, null=True) + failure_frequency = models.FloatField(default=0) # failure days per year + failure_duration = models.IntegerField(default=30) # minutes @classmethod def from_default(cls, default: QMRATreatment, risk_assessment): diff --git a/qmra/risk_assessment/risk.py b/qmra/risk_assessment/risk.py index eaf888b..40258e0 100644 --- a/qmra/risk_assessment/risk.py +++ b/qmra/risk_assessment/risk.py @@ -6,51 +6,162 @@ from qmra.risk_assessment.qmra_models import PathogenGroup, QMRAPathogens -def get_annual_risk( - inflow_min: float, inflow_max: float, - log_removal: float, - volume_per_event: float, - events_per_year: int, - distribution, - n_events: int = 10_000, - n_years: int = 1000, -): +N_SAMPLES = 10_000 # inflow concentration samples per reference pathogen +N_YEARS = 1000 # simulated years +BEST_CASE = "maximum_lrv" # result fields of best-case, which uses every step's maximum LRV +WORST_CASE = "minimum_lrv" # result fields of worst-case, which uses every step's minimum LRV +MINUTES_PER_DAY = 1440 # failure durations are given in minutes within one day +FAILURE_SEED = 2019 # failure days have their own generator, independent of the concentration samples +INFECTION_REFERENCE_LEVEL = 10 ** -4 # annual probability of infection +DALYS_REFERENCE_LEVEL = 10 ** -6 # DALYs per person per year + + +def sample_exposure_events(inflow_min: float, inflow_max: float, events_per_year: int, + n_samples: int = N_SAMPLES, n_years: int = N_YEARS): + """Log10 inflow concentration samples, and for each simulated exposure event the position of the sample it uses. + + Both cases of a scenario use the same samples and events, so they differ only by their LRVs. + """ generator = np.random.default_rng(seed=42) - inflow = generator.normal( + log_concentrations = generator.normal( loc=(np.log10(inflow_min + 10 ** (-8)) + np.log10(inflow_max)) / 2, scale=(np.log10(inflow_max) - np.log10(inflow_min + 10 ** (-8))) / 4, - size=n_events + size=n_samples ) - lrv = np.ones((n_events,)) * log_removal - outflow = inflow - lrv - dose = (10 ** outflow) * volume_per_event - event_probs = distribution.pdf(dose) - event_samples = generator.choice(event_probs, size=(n_years, events_per_year), replace=True) - return 1 - np.prod(1 - event_samples, axis=1) + sample_positions = generator.choice(n_samples, size=(n_years, events_per_year), replace=True) + return log_concentrations, sample_positions -def lrv_by_pathogen_group(treatments: Iterable[Treatment]) -> dict: - lrvs = { - PathogenGroup.Bacteria: dict(min=0, max=0), - PathogenGroup.Viruses: dict(min=0, max=0), - PathogenGroup.Protozoa: dict(min=0, max=0) - } +def draw_failing_events(treatments: list[Treatment], events_per_year: int, + n_years: int = N_YEARS) -> list[np.ndarray | None]: + """For each treatment step, a mask of the simulated exposure events that fall on one of its failure days. + + An event falls on a failure day with probability failure frequency / 365, independently per step (ADR-0001). + A step without failures gets None and no draws. Each step draws from its own stream, keyed by its position in + the train, so raising one step's failure frequency only adds failure days and leaves the other steps' as they are. + """ + failing_events = [] + for position, t in enumerate(treatments): + if t.failure_frequency > 0: + draws = np.random.default_rng([FAILURE_SEED, position]).random((n_years, events_per_year)) + failing_events.append(draws < t.failure_frequency / 365) + else: + failing_events.append(None) + return failing_events + + +def failing_steps(treatments: list[Treatment], failing_events: list[np.ndarray | None], group: PathogenGroup, + bound: str) -> list[tuple[np.ndarray, float, int]]: + """The steps that fail on some days and then lose LRV for this pathogen group, as (their failing events, LRV + lost, failure duration). A step whose minimum or maximum LRV is 0 or below loses nothing when it fails (D4).""" + steps = [] + for t, failing in zip(treatments, failing_events): + lost = step_lrv(t, group, bound) + if failing is not None and lost > 0: + steps.append((failing, lost, t.failure_duration)) + return steps + + +def worst_case_event_lrvs(worst_case_lrv: float, treatments: list[Treatment], failing_events: list[np.ndarray | None], + group: PathogenGroup, shape: tuple) -> np.ndarray: + """Worst-case LRV of each exposure event: on a failure day the failing steps' positive minimum LRVs are lost for + the whole day, whatever the failure duration.""" + event_lrvs = np.full(shape, worst_case_lrv, dtype=float) + for failing, lost, _ in failing_steps(treatments, failing_events, group, "min"): + event_lrvs[failing] -= lost + return event_lrvs + + +def mixed_water_lrv_loss(failures: list[tuple[float, int]]) -> float: + """How much LRV the mixed water of a failure day loses in best-case (CONTEXT.md, Combined failure). + + `failures` holds (LRV lost, failure duration in minutes) of each step that fails that day and loses LRV for the + pathogen group. The consumed water mixes water treated during the failure events and in normal operation, in + proportion to the failure durations. Failure events that fit into one day do not overlap; two longer than a day + in total overlap only by the minutes beyond the day. Three or more longer than a day in total count like + worst-case: all these steps are down for the whole day. + """ + total_duration = sum(duration for _, duration in failures) + all_lost = sum(lost for lost, _ in failures) + if len(failures) >= 3 and total_duration > MINUTES_PER_DAY: + return all_lost + overlap = max(0, total_duration - MINUTES_PER_DAY) + segments = [(lost, duration - overlap) for lost, duration in failures] + if overlap: + segments.append((all_lost, overlap)) + normal_minutes = MINUTES_PER_DAY - sum(minutes for _, minutes in segments) + relative_concentration = normal_minutes + sum(minutes * 10 ** lost for lost, minutes in segments) + return np.log10(relative_concentration / MINUTES_PER_DAY) + - def zero_if_none(x): return x if x is not None else 0 +def best_case_event_lrvs(best_case_lrv: float, treatments: list[Treatment], failing_events: list[np.ndarray | None], + group: PathogenGroup, shape: tuple) -> np.ndarray: + """Best-case LRV of each exposure event: on a failure day, the LRV of the mixed water under the mixed-water + assumption, from the steps that lose their positive maximum LRV for this pathogen group. Each combination of + failing steps is calculated once.""" + event_lrvs = np.full(shape, best_case_lrv, dtype=float) + steps = failing_steps(treatments, failing_events, group, "max") + if not steps: + return event_lrvs + # which of these steps fail on each event's day, as bits of one number + combination = np.zeros(shape, dtype=np.int64) + for bit, (failing, _, _) in enumerate(steps): + combination |= failing.astype(np.int64) << bit + for combination_code in np.unique(combination): + if combination_code == 0: + continue + failing_that_day = [(lost, duration) for bit, (_, lost, duration) in enumerate(steps) + if combination_code >> bit & 1] + event_lrvs[combination == combination_code] = best_case_lrv - mixed_water_lrv_loss(failing_that_day) + return event_lrvs + +def get_annual_risk(log_concentrations: np.ndarray, sample_positions: np.ndarray, event_lrvs: np.ndarray, + volume_per_event: float, distribution) -> np.ndarray: + """Annual probability of infection of each simulated year; event_lrvs holds the LRV of each exposure event.""" + # the dose-response is evaluated once per distinct LRV and concentration sample, then looked up per event + lrvs, lrv_positions = np.unique(event_lrvs, return_inverse=True) + dose = (10 ** (log_concentrations[np.newaxis, :] - lrvs[:, np.newaxis])) * volume_per_event + event_probs = distribution.pdf(dose)[lrv_positions.reshape(sample_positions.shape), sample_positions] + return 1 - np.prod(1 - event_probs, axis=1) + + +def summarize(annual_risks: np.ndarray) -> dict: + return dict( + min=annual_risks.min(), + max=annual_risks.max(), + q1=np.percentile(annual_risks, 25), + q3=np.percentile(annual_risks, 75), + median=np.median(annual_risks), + ) + + +def reference_level_exceedance(best_case_mean: float, worst_case_mean: float, reference_level: float) -> str: + """'max' if even best-case exceeds the health-based reference level, 'min' if only worst-case does.""" + return "max" if best_case_mean > reference_level else ("min" if worst_case_mean > reference_level else "none") + + +def step_lrv(treatment: Treatment, group: PathogenGroup, bound: str) -> float: + """A step's minimum or maximum LRV for a pathogen group; an empty LRV counts as 0.""" + lrv = getattr(treatment, f"{group.lower()}_{bound}") + return lrv if lrv is not None else 0 + + +def lrv_by_pathogen_group(treatments: Iterable[Treatment]) -> dict: + lrvs = {group: dict(min=0, max=0) for group in PathogenGroup} for t in treatments: - lrvs[PathogenGroup.Bacteria]["min"] += zero_if_none(t.bacteria_min) - lrvs[PathogenGroup.Bacteria]["max"] += zero_if_none(t.bacteria_max) - lrvs[PathogenGroup.Viruses]["min"] += zero_if_none(t.viruses_min) - lrvs[PathogenGroup.Viruses]["max"] += zero_if_none(t.viruses_max) - lrvs[PathogenGroup.Protozoa]["min"] += zero_if_none(t.protozoa_min) - lrvs[PathogenGroup.Protozoa]["max"] += zero_if_none(t.protozoa_max) + for group in PathogenGroup: + for bound in ["min", "max"]: + lrvs[group][bound] += step_lrv(t, group, bound) return lrvs def assess_risk(risk_assessment: RiskAssessment, inflows, treatments, save=True) -> dict[str, RiskAssessmentResult]: # assuming the model has been already validated + treatments = list(treatments) lrvs = lrv_by_pathogen_group(treatments) + # the same failure days for both cases and all reference pathogens + failing_events = draw_failing_events(treatments, risk_assessment.events_per_year) results = {} for inflow in inflows: @@ -62,59 +173,31 @@ def assess_risk(risk_assessment: RiskAssessment, inflows, treatments, save=True) def to_dalys(pr, pat=pathogen): return pr * pat.infection_to_illness * pat.dalys_per_case - # min / max probs - min_prob = get_annual_risk( - inflow.min, inflow.max, - lrvs[group]["max"], - risk_assessment.volume_per_event, risk_assessment.events_per_year, - dist - ) - max_prob = get_annual_risk( - inflow.min, inflow.max, - lrvs[group]["min"], - risk_assessment.volume_per_event, risk_assessment.events_per_year, - dist - ) - # get the stats - min_prob_mean = min_prob.mean() - max_prob_mean = max_prob.mean() - maximum_lrv_min = min_prob.min() - maximum_lrv_max = min_prob.max() - maximum_lrv_q1 = np.percentile(min_prob, 25) - maximum_lrv_q3 = np.percentile(min_prob, 75) - maximum_lrv_median = np.median(min_prob) - minimum_lrv_min = max_prob.min() - minimum_lrv_max = max_prob.max() - minimum_lrv_q1 = np.percentile(max_prob, 25) - minimum_lrv_q3 = np.percentile(max_prob, 75) - minimum_lrv_median = np.median(max_prob) - # make result + log_concentrations, sample_positions = sample_exposure_events( + inflow.min, inflow.max, risk_assessment.events_per_year) + event_lrvs = { + BEST_CASE: best_case_event_lrvs(lrvs[group]["max"], treatments, failing_events, group, + sample_positions.shape), + WORST_CASE: worst_case_event_lrvs(lrvs[group]["min"], treatments, failing_events, group, + sample_positions.shape), + } + cases = {case: get_annual_risk(log_concentrations, sample_positions, lrvs_of_case, + risk_assessment.volume_per_event, dist) + for case, lrvs_of_case in event_lrvs.items()} + stats = {} + for case, annual_risks in cases.items(): + for stat, value in summarize(annual_risks).items(): + stats[f"infection_{case}_{stat}"] = value + stats[f"dalys_{case}_{stat}"] = to_dalys(value) + best_case_mean = cases[BEST_CASE].mean() + worst_case_mean = cases[WORST_CASE].mean() results[inflow.pathogen] = RiskAssessmentResult( risk_assessment=risk_assessment, - infection_risk="max" if min_prob_mean > (10 ** -4) else ("min" if max_prob_mean > (10 ** -4) else "none"), - dalys_risk="max" if to_dalys(min_prob_mean) > (10 ** -6) else ( - "min" if to_dalys(max_prob_mean) > (10 ** -6) else "none"), + infection_risk=reference_level_exceedance(best_case_mean, worst_case_mean, INFECTION_REFERENCE_LEVEL), + dalys_risk=reference_level_exceedance(to_dalys(best_case_mean), to_dalys(worst_case_mean), + DALYS_REFERENCE_LEVEL), pathogen=inflow.pathogen, - infection_maximum_lrv_min=maximum_lrv_min, - infection_maximum_lrv_max=maximum_lrv_max, - infection_maximum_lrv_q1=maximum_lrv_q1, - infection_maximum_lrv_q3=maximum_lrv_q3, - infection_maximum_lrv_median=maximum_lrv_median, - infection_minimum_lrv_min=minimum_lrv_min, - infection_minimum_lrv_max=minimum_lrv_max, - infection_minimum_lrv_q1=minimum_lrv_q1, - infection_minimum_lrv_q3=minimum_lrv_q3, - infection_minimum_lrv_median=minimum_lrv_median, - dalys_maximum_lrv_min=to_dalys(maximum_lrv_min), - dalys_maximum_lrv_max=to_dalys(maximum_lrv_max), - dalys_maximum_lrv_q1=to_dalys(maximum_lrv_q1), - dalys_maximum_lrv_q3=to_dalys(maximum_lrv_q3), - dalys_maximum_lrv_median=to_dalys(maximum_lrv_median), - dalys_minimum_lrv_min=to_dalys(minimum_lrv_min), - dalys_minimum_lrv_max=to_dalys(minimum_lrv_max), - dalys_minimum_lrv_q1=to_dalys(minimum_lrv_q1), - dalys_minimum_lrv_q3=to_dalys(minimum_lrv_q3), - dalys_minimum_lrv_median=to_dalys(minimum_lrv_median), + **stats, ) if save: results[inflow.pathogen].save() diff --git a/qmra/risk_assessment/templates/risk-assessment-form-js.html b/qmra/risk_assessment/templates/risk-assessment-form-js.html index c414676..403bc5f 100644 --- a/qmra/risk_assessment/templates/risk-assessment-form-js.html +++ b/qmra/risk_assessment/templates/risk-assessment-form-js.html @@ -1,6 +1,57 @@ {% load static %} + + + + + + +
+
+

Result

+ +
+
The risk of this water reuse scenario can be compared to + two commonly applied health-based targets (WHO 2022): +
+
    +
  1. 1 in 10,000 infections per person per year
    2. +
  2. 0.000001 Disability Adjusted Life Years (DALY)
    3. +
+ +
+ The estimated probability of infection per year / DALYs pppy + from one or more pathogens exceeds the tolerable risk level + indicated by the WHO calculated using the minimum LRV. +
+
+ A closer look into the individual treatments + is necessary to evaluate the risk of the scenario. + +
+ +
+ + + +
+ +
+ + +
+ +
+ + + +
+ + +
+ + + \ No newline at end of file diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv new file mode 100644 index 0000000..36ceeb2 --- /dev/null +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv @@ -0,0 +1,12 @@ +"Risk Assessment's Name:","Reference assessment" +"Description:","Benchmark for the export package" + +"Result's Summary:" +,,Infection prob.,Infection prob.,Infection prob.,Infection prob.,Infection prob.,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy +pathogen,stat,min,25%,50%,75%,max,min,25%,50%,75%,max +Rotavirus,Maximum LRV,9.572485276665432e-07,1.4836628104142324e-06,1.7097025438728153e-06,1.931930578930041e-06,2.893425146477391e-06,6.700739693665803e-09,1.0385639672899628e-08,1.1967917807109707e-08,1.3523514052510289e-08,2.0253976025341736e-08 +Rotavirus,Minimum LRV,0.3640096391082689,0.41122481268750266,0.4267809590038193,0.44364273411319005,0.5027473445430916,0.002548067473757882,0.002878573688812519,0.002987466713026735,0.0031054991387923303,0.003519231411801641 +Campylobacter jejuni,Maximum LRV,4.5133408121955654e-10,2.2749717942804892e-09,3.0147686569392818e-09,3.744822868467779e-09,7.008461921920173e-09,6.22841032082988e-13,3.139461076107075e-12,4.160380746576208e-12,5.167855558485534e-12,9.671677452249837e-12 +Campylobacter jejuni,Minimum LRV,0.0001083267586116099,0.0005457054173893527,0.0007230268318788591,0.0008980393643011064,0.0016799226351181806,1.4949092688402165e-07,7.530734759973068e-07,9.977770279928255e-07,1.2392943227355267e-06,2.3182932364630894e-06 +Cryptosporidium parvum,Maximum LRV,9.220223140538053e-09,1.429064969138416e-08,1.6467869567993176e-08,1.860837314793251e-08,2.7869510144107323e-08,9.681234297564956e-12,1.5005182175953368e-11,1.7291263046392836e-11,1.9538791805329135e-11,2.926298565131269e-11 +Cryptosporidium parvum,Minimum LRV,0.0028129655229737205,0.009066386874888782,0.011592701982251985,0.01415027039915931,0.025193602188789632,2.9536137991224066e-06,9.51970621863322e-06,1.2172337081364584e-05,1.4857783919117274e-05,2.645328229822911e-05 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv new file mode 100644 index 0000000..d2a8fbf --- /dev/null +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv @@ -0,0 +1,4 @@ +Pathogen,Minimum Concentration,Maximum Concentration +Rotavirus,0.01,0.1 +Campylobacter jejuni,0.1,1.0 +Cryptosporidium parvum,0.01,0.1 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv new file mode 100644 index 0000000..f77a540 --- /dev/null +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv @@ -0,0 +1,7 @@ +Treatment,Pathogen group,Maximum LRV,Minimum LRV,Failure frequency (days per year),Failure duration (minutes) +Slow sand filtration,Viruses,4.0,0.25,0.0,30 +Slow sand filtration,Bacteria,6.0,2.0,0.0,30 +Slow sand filtration,Protozoa,5.0,0.3,0.0,30 +"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0,10.0,60 +"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6,10.0,60 +"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5,10.0,60 diff --git a/qmra/risk_assessment/tests/test_assess_risk.py b/qmra/risk_assessment/tests/test_assess_risk.py index 7ea5848..a4729b6 100644 --- a/qmra/risk_assessment/tests/test_assess_risk.py +++ b/qmra/risk_assessment/tests/test_assess_risk.py @@ -1,6 +1,8 @@ """test computation of risk assessment""" import warnings +import numpy as np + from django.core.management import call_command from django.test import TestCase from assertpy import assert_that @@ -238,4 +240,215 @@ def test_regression_test(self): .is_close_to(getattr(expected_parvum, attr), tolerance=accepted_tolerance) except AssertionError as e: failed += str(e) + "\n" - warnings.warn(failed) \ No newline at end of file + warnings.warn(failed) + + def test_same_scenario_gives_identical_results(self): + given_ra = RiskAssessment(events_per_year=365, volume_per_event=1) + given_inflows = [ + Inflow(risk_assessment=given_ra, pathogen=inflow.pathogen.name, min=inflow.min, max=inflow.max) + for inflow in QMRAInflows.get("groundwater") + ] + given_treatments = [ + Treatment(risk_assessment=given_ra, name="Primary treatment", + bacteria_min=0, bacteria_max=0.5, viruses_min=0, viruses_max=0.1, protozoa_min=0, protozoa_max=1) + ] + + def calculate(): + results = assess_risk(given_ra, given_inflows, given_treatments, save=False) + return {pathogen: r.as_dict() for pathogen, r in results.items()} + + assert_that(calculate()).is_equal_to(calculate()) + + +STATISTICS = ["min", "q1", "median", "q3", "max"] +EXCEEDANCE_ORDER = ["none", "min", "max"] +PATHOGENS = ["Rotavirus", "Campylobacter jejuni", "Cryptosporidium parvum"] + + +def calculate(steps: list[dict], events_per_year=365) -> dict[str, dict]: + """The results of a scenario with these treatment steps, as returned by the calculation.""" + given_ra = RiskAssessment(events_per_year=events_per_year, volume_per_event=1) + given_inflows = [Inflow(risk_assessment=given_ra, pathogen=p, min=0.1, max=10) for p in PATHOGENS] + given_treatments = [Treatment(risk_assessment=given_ra, name=f"step {i}", **step) for i, step in enumerate(steps)] + results = assess_risk(given_ra, given_inflows, given_treatments, save=False) + return {pathogen: r.as_dict() for pathogen, r in results.items()} + + +def case_statistics(result: dict, case: str) -> list[float]: + """Both risk measures of one case: 'minimum_lrv' is worst-case, 'maximum_lrv' is best-case.""" + return [result[f"{measure}_{case}_{stat}"] for measure in ["infection", "dalys"] for stat in STATISTICS] + + +# made-up steps whose LRVs keep the risk well below 1, so failures change the results visibly +STRONG_STEP = dict(bacteria_min=3, bacteria_max=4, viruses_min=3, viruses_max=4, protozoa_min=3, protozoa_max=4) +WEAK_STEP = dict(bacteria_min=2, bacteria_max=3, viruses_min=2, viruses_max=3, protozoa_min=2, protozoa_max=3) + + +class TestFailureDaysWorstCase(TestCase): + + def test_failure_frequency_0_gives_the_results_without_failure_inputs(self): + without = calculate([STRONG_STEP, WEAK_STEP]) + with_zero = calculate([dict(STRONG_STEP, failure_frequency=0, failure_duration=600), + dict(WEAK_STEP, failure_frequency=0, failure_duration=1)]) + + assert_that(with_zero).is_equal_to(without) + + def test_always_failing_step_loses_its_positive_minimum_lrvs_in_worst_case(self): + failing = calculate([dict(STRONG_STEP, failure_frequency=365), WEAK_STEP]) + without_min_lrvs = calculate([dict(STRONG_STEP, bacteria_min=0, viruses_min=0, protozoa_min=0), WEAK_STEP]) + + for pathogen in PATHOGENS: + with self.subTest(pathogen): + assert_that(case_statistics(failing[pathogen], "minimum_lrv")).is_equal_to( + case_statistics(without_min_lrvs[pathogen], "minimum_lrv")) + + def test_higher_failure_frequency_never_lowers_the_risk(self): + results = [calculate([dict(STRONG_STEP, failure_frequency=f), WEAK_STEP]) for f in [0, 1, 10, 100, 365]] + + for pathogen in PATHOGENS: + for step, (lower, higher) in enumerate(zip(results, results[1:])): + with self.subTest(pathogen=pathogen, step=step): + for case in ["minimum_lrv", "maximum_lrv"]: + for low, high in zip(case_statistics(lower[pathogen], case), + case_statistics(higher[pathogen], case)): + assert_that(high).is_greater_than_or_equal_to(low) + for exceedance in ["infection_risk", "dalys_risk"]: + assert_that(EXCEEDANCE_ORDER.index(higher[pathogen][exceedance])).is_greater_than_or_equal_to( + EXCEEDANCE_ORDER.index(lower[pathogen][exceedance])) + worst_median = [r["Rotavirus"]["infection_minimum_lrv_median"] for r in results] + assert_that(worst_median[-1]).is_greater_than(worst_median[0]) + + def test_failure_duration_does_not_change_worst_case(self): + short = calculate([dict(STRONG_STEP, failure_frequency=50, failure_duration=1), WEAK_STEP]) + long = calculate([dict(STRONG_STEP, failure_frequency=50, failure_duration=1440), WEAK_STEP]) + + for pathogen in PATHOGENS: + assert_that(case_statistics(long[pathogen], "minimum_lrv")).is_equal_to( + case_statistics(short[pathogen], "minimum_lrv")) + + def test_worst_case_is_never_below_best_case(self): + for frequencies in [(0, 0), (0, 5), (20, 5), (365, 5)]: + results = calculate([dict(STRONG_STEP, failure_frequency=frequencies[0]), + dict(WEAK_STEP, failure_frequency=frequencies[1])]) + for pathogen in PATHOGENS: + with self.subTest(frequencies=frequencies, pathogen=pathogen): + for worst, best in zip(case_statistics(results[pathogen], "minimum_lrv"), + case_statistics(results[pathogen], "maximum_lrv")): + assert_that(worst).is_greater_than_or_equal_to(best) + + def test_failure_removes_only_positive_lrvs(self): + # bacteria regrow at this step, viruses are removed, protozoa pass unchanged + step = dict(bacteria_min=-1, bacteria_max=-0.5, viruses_min=3, viruses_max=4, protozoa_min=0, protozoa_max=0) + normal = calculate([step, WEAK_STEP]) + failing = calculate([dict(step, failure_frequency=365), WEAK_STEP]) + + assert_that(failing["Campylobacter jejuni"]).is_equal_to(normal["Campylobacter jejuni"]) + assert_that(failing["Cryptosporidium parvum"]).is_equal_to(normal["Cryptosporidium parvum"]) + assert_that(failing["Rotavirus"]["infection_minimum_lrv_median"]).is_greater_than( + normal["Rotavirus"]["infection_minimum_lrv_median"]) + + def test_same_scenario_with_failures_gives_identical_results(self): + steps = [dict(STRONG_STEP, failure_frequency=12.5), dict(WEAK_STEP, failure_frequency=40)] + + assert_that(calculate(steps, events_per_year=20)).is_equal_to(calculate(steps, events_per_year=20)) + + +def all_groups(lrv_min: float, lrv_max: float) -> dict: + return dict(bacteria_min=lrv_min, bacteria_max=lrv_max, viruses_min=lrv_min, viruses_max=lrv_max, + protozoa_min=lrv_min, protozoa_max=lrv_max) + + +def mixed_lrv(best_case_lrv: float, segments: list[tuple[float, float]]) -> float: + """By hand: -log10 of the time-weighted mixed concentration, from (minutes, LRV lost) segments of one day.""" + idle = 1440 - sum(minutes for minutes, _ in segments) + concentration = idle * 10 ** -best_case_lrv + sum(minutes * 10 ** -(best_case_lrv - lost) + for minutes, lost in segments) + return -np.log10(concentration / 1440) + + +def assert_best_case_close(results: dict, reference: dict, pathogens: list[str]): + for pathogen in pathogens: + for value, expected in zip(case_statistics(results[pathogen], "maximum_lrv"), + case_statistics(reference[pathogen], "maximum_lrv")): + assert_that(value).described_as(pathogen).is_close_to(expected, abs(expected) * 1e-9) + + +# a step that never fails, so that the train keeps some removal on every failure day +BASE_STEP = all_groups(3, 4) + + +class TestMixedWaterBestCase(TestCase): + + def test_always_failing_full_day_loses_the_positive_maximum_lrvs_in_best_case(self): + failing = calculate([dict(STRONG_STEP, failure_frequency=365, failure_duration=1440), WEAK_STEP]) + without_max_lrvs = calculate([dict(STRONG_STEP, bacteria_max=0, viruses_max=0, protozoa_max=0), WEAK_STEP]) + + assert_best_case_close(failing, without_max_lrvs, PATHOGENS) + + def test_always_failing_part_of_the_day_gives_the_mixed_lrv_of_eq_5(self): + failing = calculate([dict(all_groups(0, 4), failure_frequency=365, failure_duration=60), BASE_STEP]) + eq_5 = mixed_lrv(8, [(60, 4)]) + reference = calculate([all_groups(0, eq_5)]) + + assert_best_case_close(failing, reference, PATHOGENS) + + def test_two_steps_over_a_day_overlap_by_the_extra_minutes(self): + failing = calculate([dict(all_groups(0, 4), failure_frequency=365, failure_duration=900), + dict(all_groups(0, 3), failure_frequency=365, failure_duration=900), BASE_STEP]) + # 360 minutes both down, 540 minutes each alone + overlap_360 = mixed_lrv(11, [(360, 7), (540, 4), (540, 3)]) + reference = calculate([all_groups(0, overlap_360)]) + + assert_best_case_close(failing, reference, PATHOGENS) + + def test_three_steps_within_a_day_do_not_overlap(self): + failing = calculate([dict(all_groups(0, 1), failure_frequency=365, failure_duration=400), + dict(all_groups(0, 1), failure_frequency=365, failure_duration=400), + dict(all_groups(0, 3), failure_frequency=365, failure_duration=400), BASE_STEP]) + no_overlap = mixed_lrv(9, [(400, 1), (400, 1), (400, 3)]) + reference = calculate([all_groups(0, no_overlap)]) + + assert_best_case_close(failing, reference, PATHOGENS) + + def test_three_steps_over_a_day_count_like_worst_case(self): + failing = calculate([dict(all_groups(0, 1), failure_frequency=365, failure_duration=800), + dict(all_groups(0, 1), failure_frequency=365, failure_duration=800), + dict(all_groups(0, 3), failure_frequency=365, failure_duration=400), BASE_STEP]) + # all three steps down for the whole day: only the step that never fails is left + reference = calculate([BASE_STEP]) + + assert_best_case_close(failing, reference, PATHOGENS) + + def test_only_steps_that_lose_lrv_for_the_pathogen_group_count(self): + # the third step removes no protozoa, so for protozoa this is a day with two failing steps + steps = [all_groups(0, 1), all_groups(0, 1), dict(all_groups(0, 1), protozoa_max=0)] + failing = calculate([dict(step, failure_frequency=365, failure_duration=900) for step in steps] + [BASE_STEP]) + two_steps_overlapping = mixed_lrv(6, [(360, 2), (540, 1), (540, 1)]) + reference = calculate([dict(all_groups(3, 4), protozoa_min=0, protozoa_max=two_steps_overlapping)]) + + assert_best_case_close(failing, reference, PATHOGENS) + + def test_three_steps_crossing_a_day_raise_best_case(self): + within, over = [calculate([dict(all_groups(0, 1), failure_frequency=100, failure_duration=duration)] * 3 + [BASE_STEP]) + for duration in [480, 481]] + + for pathogen in PATHOGENS: + for before, after in zip(case_statistics(within[pathogen], "maximum_lrv"), + case_statistics(over[pathogen], "maximum_lrv")): + assert_that(after).is_greater_than_or_equal_to(before) + assert_that(over["Rotavirus"]["infection_maximum_lrv_median"]).is_greater_than( + within["Rotavirus"]["infection_maximum_lrv_median"]) + + def test_longer_failure_duration_never_lowers_best_case(self): + results = [calculate([dict(STRONG_STEP, failure_frequency=100, failure_duration=duration), + dict(WEAK_STEP, failure_frequency=100, failure_duration=duration)]) + for duration in [1, 60, 600, 1000, 1440]] + + for pathogen in PATHOGENS: + for pair, (shorter, longer) in enumerate(zip(results, results[1:])): + with self.subTest(pathogen=pathogen, pair=pair): + for short, long in zip(case_statistics(shorter[pathogen], "maximum_lrv"), + case_statistics(longer[pathogen], "maximum_lrv")): + assert_that(long).is_greater_than_or_equal_to(short) + best_median = [r["Rotavirus"]["infection_maximum_lrv_median"] for r in results] + assert_that(best_median[-1]).is_greater_than(best_median[0]) diff --git a/qmra/risk_assessment/tests/test_failure_inputs.py b/qmra/risk_assessment/tests/test_failure_inputs.py new file mode 100644 index 0000000..f5517a0 --- /dev/null +++ b/qmra/risk_assessment/tests/test_failure_inputs.py @@ -0,0 +1,495 @@ +"""failure inputs of treatment steps, tested over HTTP like a browser uses the configurator""" +import io +import re +from html.parser import HTMLParser +from zipfile import ZipFile + +import pandas as pd +from assertpy import assert_that +from django.test import TestCase +from django.urls import reverse + +from qmra.risk_assessment.models import RiskAssessment, Inflow, Treatment +from qmra.risk_assessment.user_models import UserTreatment +from qmra.user.models import User + + +class ConfiguratorFormParser(HTMLParser): + """Collects what a browser would submit from the configurator form.""" + + def __init__(self): + super().__init__() + self.data = {} + self.in_configurator = False + self.select_name = None + self.first_option = None + + def handle_starttag(self, tag, attrs): + attrs = dict(attrs) + if tag == "form": + self.in_configurator = attrs.get("id") == "configurator" + if not self.in_configurator: + return + if tag == "input" and attrs.get("name"): + if attrs.get("type") in ("submit", "button"): + return + if attrs.get("type") == "checkbox" and "checked" not in attrs: + return + self.data[attrs["name"]] = attrs.get("value", "") + elif tag == "select": + self.select_name = attrs.get("name") + self.first_option = None + elif tag == "option" and self.select_name: + if self.first_option is None: + self.first_option = attrs.get("value", "") + self.data.setdefault(self.select_name, self.first_option) + if "selected" in attrs: + self.data[self.select_name] = attrs.get("value", "") + + def handle_endtag(self, tag): + if tag == "form": + self.in_configurator = False + elif tag == "select": + self.select_name = None + + +def configurator_data(response) -> dict: + parser = ConfiguratorFormParser() + parser.feed(response.content.decode()) + return parser.data + + +def field_errors(response, name: str) -> list[str]: + """The error messages the configurator shows below the input with this name.""" + return re.findall(rf'(.*?)', + response.content.decode()) + + +def add_step(data: dict, name: str, **values): + """Add a treatment step card the way the configurator's script does: copy the empty form.""" + n = int(data["treatments-TOTAL_FORMS"]) + for key, value in list(data.items()): + if key.startswith("treatments-__prefix__-") and not key.endswith("-DELETE"): + data[key.replace("__prefix__", str(n))] = value + data[f"treatments-{n}-name"] = name + for field, value in values.items(): + data[f"treatments-{n}-{field}"] = value + data["treatments-TOTAL_FORMS"] = str(n + 1) + + +def failure_inputs_per_step(data: dict) -> list[dict]: + """The treatment step cards of a configurator, in order, with their failure inputs.""" + return [ + dict(name=data[f"treatments-{i}-name"], + failure_frequency=float(data[f"treatments-{i}-failure_frequency"]), + failure_duration=int(data[f"treatments-{i}-failure_duration"])) + for i in range(int(data["treatments-TOTAL_FORMS"])) + ] + + +def new_configurator(client) -> dict: + """A new configurator with groundwater, drinking water and no treatment steps yet.""" + data = configurator_data(client.get(reverse("assessment"))) + for i in range(3): + data[f"inflow-{i}-min"] = "10" + data[f"inflow-{i}-max"] = "100" + data["ra-source_name"] = "groundwater" + data["ra-exposure_name"] = "drinking water" + data["ra-events_per_year"] = "365" + data["ra-volume_per_event"] = "1" + return data + + +TREATMENTS_CSV = "exposure-assessment/treatments.csv" + + +def layout(csv: bytes) -> list[str]: + """The lines of an exported table with every number replaced, so only its rows and columns remain.""" + return [re.sub(r"-?\d+(\.\d+)?(e[-+]?\d+)?", "#", line) for line in csv.decode().splitlines()] + + +# high enough that failures change the results visibly instead of leaving them at a risk of 1 +HIGH_LRVS = dict(bacteria_min=3, bacteria_max=4, viruses_min=3, viruses_max=4, protozoa_min=3, protozoa_max=4) +LRVS = dict(bacteria_min=1, bacteria_max=2, viruses_min=1, viruses_max=2, protozoa_min=1, protozoa_max=2) + + +class LoggedInTestCase(TestCase): + + def setUp(self): + self.user = User.objects.create_user("test-user", "test-user@test.com", "password") + self.client.force_login(self.user) + + def save_new(self, data: dict) -> RiskAssessment: + response = self.client.post(reverse("assessment"), data) + assert_that(response.status_code).is_equal_to(302) + return RiskAssessment.objects.get(user=self.user) + + +class TestFailureInputs(LoggedInTestCase): + + def post_configurator(self, data: dict, assessment: RiskAssessment = None): + url = reverse("assessment") if assessment is None else reverse("assessment", args=[assessment.id]) + return self.client.post(url, data) + + def reopen(self, assessment: RiskAssessment) -> dict: + return configurator_data(self.client.get(reverse("assessment", args=[assessment.id]))) + + def results(self, assessment: RiskAssessment) -> dict: + return {r.pathogen: r.as_dict() for r in RiskAssessment.objects.get(id=assessment.id).results.all()} + + def export(self, assessment: RiskAssessment) -> dict: + response = self.client.get(reverse("assessment-export", args=[assessment.id])) + with ZipFile(io.BytesIO(response.content)) as archive: + return {name: archive.read(name) for name in archive.namelist()} + + def test_new_step_card_shows_failure_fields_with_units_and_defaults(self): + response = self.client.get(reverse("assessment")) + + assert_that(response.content.decode()).contains( + "Failure frequency (days per year)", "Failure duration (minutes)") + data = configurator_data(response) + assert_that(float(data["treatments-__prefix__-failure_frequency"])).is_equal_to(0) + assert_that(int(data["treatments-__prefix__-failure_duration"])).is_equal_to(30) + + def test_bundled_step_starts_at_defaults(self): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **LRVS) + + assessment = self.save_new(data) + + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Primary treatment", failure_frequency=0, failure_duration=30) + ]) + + def test_failure_inputs_survive_save_and_reopen_per_step(self): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **LRVS, failure_frequency="0.5", failure_duration="15") + add_step(data, "Primary treatment", **LRVS, failure_frequency="12", failure_duration="600") + add_step(data, "Slow sand filtration", **LRVS, failure_frequency="365", failure_duration="1440") + + assessment = self.save_new(data) + + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Primary treatment", failure_frequency=0.5, failure_duration=15), + dict(name="Primary treatment", failure_frequency=12, failure_duration=600), + dict(name="Slow sand filtration", failure_frequency=365, failure_duration=1440), + ]) + + def test_editing_stores_new_values_and_values_stay_with_their_step(self): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **LRVS, failure_frequency="1", failure_duration="10") + add_step(data, "Slow sand filtration", **LRVS, failure_frequency="2", failure_duration="20") + add_step(data, "Primary treatment", **LRVS, failure_frequency="3", failure_duration="30") + assessment = self.save_new(data) + + data = self.reopen(assessment) + data["treatments-0-DELETE"] = "on" + data["treatments-2-failure_frequency"] = "4.5" + data["treatments-2-failure_duration"] = "45" + response = self.post_configurator(data, assessment) + + assert_that(response.status_code).is_equal_to(302) + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Slow sand filtration", failure_frequency=2, failure_duration=20), + dict(name="Primary treatment", failure_frequency=4.5, failure_duration=45), + ]) + + def test_out_of_range_or_fractional_values_are_rejected_with_a_message(self): + cases = [ + ("-1", "30", "failure frequency must be between 0 and 365 days per year"), + ("365.5", "30", "failure frequency must be between 0 and 365 days per year"), + ("0", "0", "failure duration must be between 1 and 1440 minutes"), + ("0", "1441", "failure duration must be between 1 and 1440 minutes"), + ("1", "2.5", "Enter a whole number."), + ("1", "", "This field is required."), + ("", "30", "This field is required."), + ] + for frequency, duration, message in cases: + with self.subTest(frequency=frequency, duration=duration): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **LRVS, + failure_frequency=frequency, failure_duration=duration) + + response = self.post_configurator(data) + + assert_that(response.status_code).is_equal_to(200) + assert_that(response.content.decode()).contains(message) + assert_that(RiskAssessment.objects.filter(user=self.user).count()).is_equal_to(0) + + def test_range_bounds_are_accepted(self): + for frequency, duration in [("0", "1"), ("365", "1440"), ("0.5", "30")]: + with self.subTest(frequency=frequency, duration=duration): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **LRVS, + failure_frequency=frequency, failure_duration=duration) + + response = self.post_configurator(data) + + assert_that(response.status_code).is_equal_to(302) + + def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): + assessment = RiskAssessment.objects.create(user=self.user, events_per_year=365, volume_per_event=1) + for pathogen in ["Rotavirus", "Campylobacter jejuni", "Cryptosporidium parvum"]: + Inflow.objects.create(risk_assessment=assessment, pathogen=pathogen, min=10, max=100) + Treatment.objects.create(risk_assessment=assessment, name="Primary treatment", **LRVS) + + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Primary treatment", failure_frequency=0, failure_duration=30) + ]) + + @staticmethod + def exported_failure_inputs(export: dict) -> list[tuple]: + """(treatment, pathogen group, failure frequency, failure duration) of each row of the exported treatment table.""" + table = pd.read_csv(io.BytesIO(export[TREATMENTS_CSV])) + return list(table[["Treatment", "Pathogen group", "Failure frequency (days per year)", + "Failure duration (minutes)"]].itertuples(index=False, name=None)) + + def saved_assessment_with_two_steps(self) -> RiskAssessment: + data = new_configurator(self.client) + add_step(data, "Primary treatment", **HIGH_LRVS) + add_step(data, "Slow sand filtration", **HIGH_LRVS) + return self.save_new(data) + + def test_failure_frequency_0_changes_neither_results_nor_export_beyond_the_durations(self): + assessment = self.saved_assessment_with_two_steps() + results_before = self.results(assessment) + export_before = self.export(assessment) + + data = self.reopen(assessment) + data["treatments-0-failure_duration"] = "120" + data["treatments-1-failure_duration"] = "1440" + self.post_configurator(data, assessment) + + assert_that(failure_inputs_per_step(self.reopen(assessment))[0]["failure_duration"]).is_equal_to(120) + assert_that(self.results(assessment)).is_equal_to(results_before) + # the export records the new durations in the treatment table; everything else stays identical + export_after = self.export(assessment) + durations = {duration for *_, duration in self.exported_failure_inputs(export_after)} + export_after.pop(TREATMENTS_CSV) + export_before.pop(TREATMENTS_CSV) + assert_that(export_after).is_equal_to(export_before) + assert_that(durations).is_equal_to({120, 1440}) + + def test_export_records_the_failure_inputs_of_each_step(self): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **HIGH_LRVS, failure_frequency="12.5", failure_duration="90") + add_step(data, "Slow sand filtration", **HIGH_LRVS) + assessment = self.save_new(data) + + groups = ["Viruses", "Bacteria", "Protozoa"] + assert_that(self.exported_failure_inputs(self.export(assessment))).is_equal_to( + [("Primary treatment", group, 12.5, 90) for group in groups] + + [("Slow sand filtration", group, 0, 30) for group in groups]) + + def test_export_keeps_its_files_and_columns_with_failures(self): + assessment = self.saved_assessment_with_two_steps() + results_before = self.results(assessment) + export_before = self.export(assessment) + + data = self.reopen(assessment) + data["treatments-0-failure_frequency"] = "20" + data["treatments-1-failure_frequency"] = "365" + self.post_configurator(data, assessment) + + assert_that(self.results(assessment)).is_not_equal_to(results_before) + export_after = self.export(assessment) + assert_that(sorted(export_after)).is_equal_to(sorted(export_before)) + for name in export_before: + if name.endswith(".csv"): + assert_that(layout(export_after[name])).described_as(name).is_equal_to(layout(export_before[name])) + + +NO_LRVS = dict(bacteria_min="", bacteria_max="", viruses_min="", viruses_max="", protozoa_min="", protozoa_max="") +D4_REJECTED = { + "empty LRVs": NO_LRVS, + "zero LRVs": dict(NO_LRVS, bacteria_min=0, bacteria_max=0, viruses_min=0, viruses_max=0), + "recontamination": dict(bacteria_min=-1, bacteria_max=-0.5, viruses_min=-1, viruses_max=-0.5, + protozoa_min=-1, protozoa_max=-0.5), +} +D4_ACCEPTED = { + "one positive maximum": dict(NO_LRVS, protozoa_min=0, protozoa_max=2), + "regrowth in one group": dict(bacteria_min=-1, bacteria_max=-0.5, viruses_min=1, viruses_max=2, + protozoa_min=1, protozoa_max=2), +} +D4_MESSAGE = "failure frequency must be 0 for a treatment step without a positive LRV" + + +class TestFailureOnlyWithPositiveLrv(LoggedInTestCase): + + def save_with_step(self, lrvs: dict, failure_frequency: str): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **lrvs, failure_frequency=failure_frequency, failure_duration="30") + return self.client.post(reverse("assessment"), data) + + def test_step_without_positive_lrv_cannot_fail(self): + for case, lrvs in D4_REJECTED.items(): + with self.subTest(case): + response = self.save_with_step(lrvs, failure_frequency="1") + + assert_that(response.status_code).is_equal_to(200) + assert_that(field_errors(response, "treatments-0-failure_frequency")).is_equal_to([D4_MESSAGE]) + assert_that(RiskAssessment.objects.filter(user=self.user).count()).is_equal_to(0) + + def test_step_without_positive_lrv_keeps_failure_frequency_0(self): + for case, lrvs in D4_REJECTED.items(): + with self.subTest(case): + response = self.save_with_step(lrvs, failure_frequency="0") + + assert_that(response.status_code).is_equal_to(302) + + def test_step_with_one_positive_lrv_can_fail(self): + for case, lrvs in D4_ACCEPTED.items(): + with self.subTest(case): + response = self.save_with_step(lrvs, failure_frequency="1") + + assert_that(response.status_code).is_equal_to(302) + + +class TestGuestFailureInputs(TestCase): + + def guest_result(self, **step_values): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **{**LRVS, **step_values}) + return self.client.post(reverse("assessment-result"), data) + + def test_guest_sees_failure_fields(self): + response = self.client.get(reverse("assessment")) + + assert_that(response.content.decode()).contains( + "Failure frequency (days per year)", "Failure duration (minutes)") + + def test_guest_result_with_failure_inputs_is_shown_and_not_stored(self): + response = self.guest_result(failure_frequency="12", failure_duration="60") + + assert_that(response.status_code).is_equal_to(200) + assert_that(RiskAssessment.objects.count()).is_equal_to(0) + assert_that(Treatment.objects.count()).is_equal_to(0) + + def test_guest_result_sends_back_messages_of_invalid_inputs(self): + cases = [ + (dict(failure_frequency="366"), "treatments-0-failure_frequency", + "failure frequency must be between 0 and 365 days per year"), + (dict(failure_duration="2.5"), "treatments-0-failure_duration", "Enter a whole number."), + (dict(bacteria_min="3", bacteria_max="1"), "treatments-0-bacteria_min", "min. must be less than max"), + ] + for step_values, field, message in cases: + with self.subTest(field=field, message=message): + response = self.guest_result(**step_values) + + assert_that(response.status_code).is_equal_to(422) + assert_that(response.json()["errors"]).contains_entry({field: [message]}) + assert_that(RiskAssessment.objects.count()).is_equal_to(0) + + def test_guest_result_rejects_failing_step_without_positive_lrv(self): + for case, lrvs in D4_REJECTED.items(): + with self.subTest(case): + response = self.guest_result(**lrvs, failure_frequency="1") + + assert_that(response.status_code).is_equal_to(422) + assert_that(response.json()["errors"]).contains_entry( + {"treatments-0-failure_frequency": [D4_MESSAGE]}) + + def test_guest_result_accepts_step_without_positive_lrv_and_failure_frequency_0(self): + for case, lrvs in D4_REJECTED.items(): + with self.subTest(case): + response = self.guest_result(**lrvs, failure_frequency="0") + + assert_that(response.status_code).is_equal_to(200) + + def test_guest_result_accepts_failing_step_with_one_positive_lrv(self): + for case, lrvs in D4_ACCEPTED.items(): + with self.subTest(case): + response = self.guest_result(**lrvs, failure_frequency="1") + + assert_that(response.status_code).is_equal_to(200) + + +def personal_step_form(response) -> str: + """The HTML of the personal treatment step form on the configurator page.""" + html = response.content.decode() + start = html.index('id="user-treatment-form"') + return html[start:html.index("", start)] + + +class TestPersonalTreatmentStepFailureInputs(LoggedInTestCase): + + def create_personal_step(self, name="Plant UV", **values): + data = dict(name=name, **LRVS, failure_frequency="0", failure_duration="30") + data.update(values) + return self.client.post(reverse("treatment"), data, HTTP_REFERER=reverse("assessment")) + + def personal_steps(self, client=None) -> dict: + return (client or self.client).get(reverse("treatments")).json() + + def failure_inputs(self, name: str) -> tuple: + step = self.personal_steps()[name] + return step["failure_frequency"], step["failure_duration"] + + def test_form_shows_failure_fields_with_units_and_defaults(self): + form = personal_step_form(self.client.get(reverse("assessment"))) + + assert_that(form).contains("Failure frequency (days per year)", "Failure duration (minutes)") + assert_that(form).matches(r'name="failure_frequency" value="0"') + assert_that(form).matches(r'name="failure_duration" value="30"') + + def test_failure_inputs_appear_in_the_owners_list(self): + response = self.create_personal_step(failure_frequency="2.5", failure_duration="90") + + assert_that(response.status_code).is_equal_to(302) + assert_that(self.failure_inputs("Plant UV")).is_equal_to((2.5, 90)) + + def test_other_users_do_not_see_the_personal_step(self): + self.create_personal_step(failure_frequency="2.5", failure_duration="90") + other = User.objects.create_user("other-user", "other-user@test.com", "password") + other_client = self.client_class() + other_client.force_login(other) + + assert_that(self.personal_steps(other_client)).does_not_contain_key("Plant UV") + + def test_invalid_values_are_rejected_with_a_message(self): + cases = [ + (dict(failure_frequency="366"), "failure frequency must be between 0 and 365 days per year"), + (dict(failure_duration="0"), "failure duration must be between 1 and 1440 minutes"), + (dict(failure_duration="2.5"), "Enter a whole number."), + (dict(failure_duration=""), "This field is required."), + (dict(NO_LRVS, failure_frequency="1"), D4_MESSAGE), + (dict(bacteria_min="3", bacteria_max="1"), "min. must be less than max"), + ] + for values, message in cases: + with self.subTest(message=message, values=values): + response = self.create_personal_step(**values) + + assert_that(response.status_code).is_equal_to(422) + assert_that(response.content.decode()).contains(message) + assert_that(self.personal_steps()).is_empty() + + def test_valid_values_including_d4_cases_are_accepted(self): + cases = { + "bounds low": dict(failure_frequency="0", failure_duration="1"), + "bounds high": dict(failure_frequency="365", failure_duration="1440"), + "no positive LRV, frequency 0": dict(NO_LRVS, failure_frequency="0"), + **{case: dict(lrvs, failure_frequency="1") for case, lrvs in D4_ACCEPTED.items()}, + } + for case, values in cases.items(): + with self.subTest(case): + response = self.create_personal_step(name=case, **values) + + assert_that(response.status_code).is_equal_to(302) + assert_that(self.personal_steps()).contains_key(case) + + def test_personal_step_from_before_failure_inputs_offers_defaults(self): + UserTreatment.objects.create(user=self.user, name="Old step", **LRVS) + + assert_that(self.failure_inputs("Old step")).is_equal_to((0, 30)) + + def test_changing_the_copy_in_an_assessment_leaves_the_personal_step_unchanged(self): + self.create_personal_step(failure_frequency="2.5", failure_duration="90") + data = new_configurator(self.client) + add_step(data, "Plant UV", **LRVS, failure_frequency="10", failure_duration="15") + + assessment = self.save_new(data) + + assert_that(failure_inputs_per_step(configurator_data( + self.client.get(reverse("assessment", args=[assessment.id]))))).is_equal_to([ + dict(name="Plant UV", failure_frequency=10, failure_duration=15)]) + assert_that(self.failure_inputs("Plant UV")).is_equal_to((2.5, 90)) diff --git a/qmra/risk_assessment/tests/test_reference_export.py b/qmra/risk_assessment/tests/test_reference_export.py new file mode 100644 index 0000000..44842e0 --- /dev/null +++ b/qmra/risk_assessment/tests/test_reference_export.py @@ -0,0 +1,103 @@ +"""the export package changes only on purpose (quality goal Export stability)""" +import io +import os +import re +import shutil +from pathlib import Path +from zipfile import ZipFile + +from assertpy import assert_that +from django.test import TestCase +from django.urls import reverse + +from qmra.risk_assessment.models import RiskAssessment, Inflow, Treatment +from qmra.user.models import User + +REFERENCE = Path(__file__).parent / "reference_export" +FILE_LIST = REFERENCE / "files.txt" +PACKAGE = REFERENCE / "package" +# the plots are only checked for presence: their rendering can differ between machines +PLOTS = ["results-plots/infection-probability.png", "results-plots/dalys-pppy.png"] +UPDATE_REFERENCE = os.getenv("UPDATE_REFERENCE_EXPORT") == "1" + + +def is_compared(name: str) -> bool: + """Whether a ZIP entry is compared by content: not a folder and not a plot.""" + return not name.endswith("/") and name not in PLOTS + + +NUMBER = re.compile(r"-?\d+(?:\.\d+)?(?:e[-+]?\d+)?") +# Tiny best-case probabilities are computed as 1 - exp(-k * dose), where most digits cancel, so machines with other +# processors or math libraries differ by up to about 1e-5 relative (seen between CI and a developer machine). +RELATIVE_TOLERANCE = 1e-3 + + +def masked(name: str, content: bytes) -> str: + """The content of an exported file as it is compared; the plots embedded in the report are masked.""" + text = content.decode("utf-8") + if name.endswith(".html"): + text, plots = re.subn(r"base64,\s*[A-Za-z0-9+/=]+", "base64, ", text) + assert_that(plots).described_as(f"plots masked in {name}").is_equal_to(len(PLOTS)) + return text + + +def assert_same_content(name: str, actual: str, expected: str): + """The text of a file identical, and each of its numbers within the relative tolerance.""" + assert_that(NUMBER.sub("#", actual)).described_as(name).is_equal_to(NUMBER.sub("#", expected)) + for value, reference in zip(NUMBER.findall(actual), NUMBER.findall(expected)): + assert_that(float(value)).described_as(f"{name}: {value} instead of {reference}").is_close_to( + float(reference), abs(float(reference)) * RELATIVE_TOLERANCE) + + +class TestReferenceExport(TestCase): + + def setUp(self): + self.user = User.objects.create_user("reference-user", "reference-user@test.com", "password") + self.client.force_login(self.user) + self.assessment = RiskAssessment.objects.create( + user=self.user, name="Reference assessment", description="Benchmark for the export package", + source_name="groundwater", exposure_name="drinking water", events_per_year=365, volume_per_event=1) + # fixed values rather than bundled data, so a data release doesn't change the reference + for pathogen, (low, high) in {"Rotavirus": (0.01, 0.1), "Campylobacter jejuni": (0.1, 1), + "Cryptosporidium parvum": (0.01, 0.1)}.items(): + Inflow.objects.create(risk_assessment=self.assessment, pathogen=pathogen, min=low, max=high) + steps = [ + dict(name="Slow sand filtration", bacteria_min=2, bacteria_max=6, viruses_min=0.25, viruses_max=4, + protozoa_min=0.3, protozoa_max=5), + dict(name="UV disinfection 20 mJ/cm2, drinking", bacteria_min=4.6, bacteria_max=6, viruses_min=1, + viruses_max=3, protozoa_min=2.5, protozoa_max=3, failure_frequency=10, failure_duration=60), + ] + for index, step in enumerate(steps): + Treatment.objects.create(risk_assessment=self.assessment, train_index=index, **step) + + def export(self) -> dict[str, bytes]: + response = self.client.get(reverse("assessment-export", args=[self.assessment.id])) + assert_that(response.status_code).is_equal_to(200) + with ZipFile(io.BytesIO(response.content)) as archive: + return {name: archive.read(name) for name in archive.namelist()} + + def update_reference(self, export: dict[str, bytes]): + shutil.rmtree(PACKAGE, ignore_errors=True) + FILE_LIST.parent.mkdir(parents=True, exist_ok=True) + FILE_LIST.write_text("".join(f"{name}\n" for name in sorted(export)), encoding="utf-8") + for name, content in export.items(): + if is_compared(name): + path = PACKAGE / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(masked(name, content), encoding="utf-8") + + def test_export_package_matches_the_reference(self): + export = self.export() + if UPDATE_REFERENCE: + self.update_reference(export) + self.skipTest("reference export updated; review the changes before committing them") + + assert_that(str(FILE_LIST)).described_as("no reference export stored yet").exists() + assert_that(sorted(export)).described_as("files of the export package").is_equal_to( + FILE_LIST.read_text(encoding="utf-8").splitlines()) + for name, content in export.items(): + if name in PLOTS: + assert_that(content).described_as(name).is_not_empty() + elif is_compared(name): + assert_that(str(PACKAGE / name)).described_as(f"reference of {name}").exists() + assert_same_content(name, masked(name, content), (PACKAGE / name).read_text(encoding="utf-8")) diff --git a/qmra/risk_assessment/tests/test_risk_assessment_form.py b/qmra/risk_assessment/tests/test_risk_assessment_form.py index e28527d..d1ae84b 100644 --- a/qmra/risk_assessment/tests/test_risk_assessment_form.py +++ b/qmra/risk_assessment/tests/test_risk_assessment_form.py @@ -87,7 +87,9 @@ def test_that_negative_are_allowed(self): viruses_min=-2, viruses_max=-1, protozoa_min=-2, - protozoa_max=-1 + protozoa_max=-1, + failure_frequency=0, + failure_duration=30, ) given_form = TreatmentForm(data=data) # ugly hack to work around dynamic choices... @@ -104,7 +106,9 @@ def test_that_min_needs_to_be_less_than_max(self): viruses_min=0, viruses_max=0, protozoa_min=0, - protozoa_max=0 + protozoa_max=0, + failure_frequency=0, + failure_duration=30, ) for prfx in ["bacteria", "viruses", "protozoa"]: mn = prfx + "_min" diff --git a/qmra/risk_assessment/user_models.py b/qmra/risk_assessment/user_models.py index 3b5f481..9e3891e 100644 --- a/qmra/risk_assessment/user_models.py +++ b/qmra/risk_assessment/user_models.py @@ -6,7 +6,7 @@ from crispy_forms.helper import FormHelper from crispy_forms.layout import Layout, Field, Row, Column, HTML, Submit -from qmra.risk_assessment.forms import _zero_if_none +from qmra.risk_assessment.forms import check_treatment_step from qmra.user.models import User @@ -138,6 +138,8 @@ class UserTreatment(models.Model): viruses_max = models.FloatField(blank=True, null=True) protozoa_min = models.FloatField(blank=True, null=True) protozoa_max = models.FloatField(blank=True, null=True) + failure_frequency = models.FloatField(default=0) # failure days per year + failure_duration = models.IntegerField(default=30) # minutes class UserTreatmentForm(forms.ModelForm): @@ -150,7 +152,9 @@ class Meta: 'viruses_min', 'viruses_max', "protozoa_min", - "protozoa_max" + "protozoa_max", + "failure_frequency", + "failure_duration", ] def __init__(self, *args, **kwargs): @@ -167,6 +171,8 @@ def __init__(self, *args, **kwargs): self.fields['viruses_max'].label = "" self.fields['protozoa_min'].label = "" self.fields['protozoa_max'].label = "" + self.fields['failure_frequency'].label = "Failure frequency (days per year)" + self.fields['failure_duration'].label = "Failure duration (minutes)" label_style = "class='text-muted text-center w-100' style='margin-top: .4em;'" self.helper.layout = Layout( Field("name"), @@ -179,22 +185,11 @@ def __init__(self, *args, **kwargs): Column("viruses_min"), Column("viruses_max")), Row(Column(HTML(f"")), Column("protozoa_min"), Column("protozoa_max")), + Row(Column("failure_frequency"), Column("failure_duration")), Submit('submit', 'Submit') ) def clean(self): cleaned_data = super().clean() - b_min = _zero_if_none(cleaned_data.get("bacteria_min", 0)) - b_max = _zero_if_none(cleaned_data.get("bacteria_max", 0)) - v_min = _zero_if_none(cleaned_data.get("viruses_min", 0)) - v_max = _zero_if_none(cleaned_data.get("viruses_max", 0)) - p_min = _zero_if_none(cleaned_data.get("protozoa_min", 0)) - p_max = _zero_if_none(cleaned_data.get("protozoa_max", 0)) - msg = "min. must be less than max" - if b_min > b_max: - self.add_error("bacteria_min", msg) - if v_min > v_max: - self.add_error("viruses_min", msg) - if p_min > p_max: - self.add_error("protozoa_min", msg) + check_treatment_step(self, cleaned_data) return cleaned_data diff --git a/qmra/risk_assessment/views.py b/qmra/risk_assessment/views.py index 2e4122e..c0babaa 100644 --- a/qmra/risk_assessment/views.py +++ b/qmra/risk_assessment/views.py @@ -2,6 +2,7 @@ from crispy_forms.utils import render_crispy_form from django.contrib.auth.decorators import login_required +from django.core.exceptions import NON_FIELD_ERRORS from django.http import HttpResponseRedirect, HttpResponse, JsonResponse from django.shortcuts import render from django.template.context_processors import csrf @@ -55,6 +56,24 @@ def assess_and_save_results(risk_assessment: RiskAssessment) -> RiskAssessment: return RiskAssessment.objects.get(id=risk_assessment.id) +def errors_by_field_name(risk_assessment_form, inflow_form, treatment_form) -> dict[str, list[str]]: + """Error messages keyed by the name of the input they belong to; the rest under NON_FIELD_ERRORS.""" + errors = {} + + def add(key, messages): + errors.setdefault(key, []).extend(messages) + + for field, messages in risk_assessment_form.errors.items(): + add(field if field == NON_FIELD_ERRORS else risk_assessment_form.add_prefix(field), messages) + for formset in [inflow_form, treatment_form]: + # formset.deleted_forms is empty for an invalid formset, so removed steps are filtered here + for form in [f for f in formset.forms if not formset._should_delete_form(f)]: + for field, messages in form.errors.items(): + add(field if field == NON_FIELD_ERRORS else form.add_prefix(field), messages) + add(NON_FIELD_ERRORS, formset.non_form_errors()) + return {key: messages for key, messages in errors.items() if messages} + + @login_required(login_url="/login") def list_risk_assessment_view(request): return render(request, "risk-assessment-list.html", @@ -181,9 +200,8 @@ def risk_assessment_result(request): for t in treatments if len(t.above_max_lrv())})) else: - # print(inflow_form.errors) - # print(treatment_form.errors) - return HttpResponse(status=422) + return JsonResponse(dict(errors=errors_by_field_name(risk_assessment_form, inflow_form, treatment_form)), + status=422) elif request.method == "GET": risk_assessment_id = request.GET.get("id") @@ -292,13 +310,13 @@ def create_treatment(request): # handle duplicate names if any(UserExposure.objects.filter(name=treatment_form.instance.name, user=request.user).all()): treatment_form.add_error("name", "'name' must be unique. You already have a treatment with this name") - ctx = {} - ctx.update(csrf(request)) - return HttpResponse(render_crispy_form(treatment_form, context=ctx), status=422) - treatment_form.instance.user = request.user - treatment_form.save(commit=True) - return HttpResponseRedirect(request.META["HTTP_REFERER"]) - return HttpResponse(status=422) + else: + treatment_form.instance.user = request.user + treatment_form.save(commit=True) + return HttpResponseRedirect(request.META["HTTP_REFERER"]) + ctx = {} + ctx.update(csrf(request)) + return HttpResponse(render_crispy_form(treatment_form, context=ctx), status=422) return HttpResponse(status=404)