From 8307a7aa6e7e329614449c51555a1c9219bda25b Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 25 Sep 2026 09:52:07 +0000 Subject: [PATCH 01/43] docs: add C1 tickets 04-06 (#21, #22, #23) [skip ci] Co-Authored-By: Claude Opus 5.5 (1M context) --- .../04-c1-failure-inputs-in-configurator.md | 24 +++++++++++++++++++ ...y-needs-positive-lrv-and-guest-messages.md | 20 ++++++++++++++++ ...-personal-treatment-step-failure-inputs.md | 24 +++++++++++++++++++ 3 files changed, 68 insertions(+) create mode 100644 docs/tickets/04-c1-failure-inputs-in-configurator.md create mode 100644 docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md create mode 100644 docs/tickets/06-c1-personal-treatment-step-failure-inputs.md diff --git a/docs/tickets/04-c1-failure-inputs-in-configurator.md b/docs/tickets/04-c1-failure-inputs-in-configurator.md new file mode 100644 index 0000000..8903651 --- /dev/null +++ b/docs/tickets/04-c1-failure-inputs-in-configurator.md @@ -0,0 +1,24 @@ +# 04: Failure inputs on a treatment step in the configurator: enter, check ranges, save, reopen, edit + +**What to build:** Every treatment step in the assessment configurator gets two more fields: **failure frequency** (failure days per year) and **failure duration** (minutes). Registered users can save an assessment with these values, and when they reopen or edit it they see the same values again. The values do not change the result yet; the calculation follows in C2. + +Tidy-up comes first. The configurator's treatment step form and the personal treatment step form both contain the same minimum ≤ maximum LRV check. It becomes one shared rule that both forms use, and the failure rules are added to it later. This Ticket also adds the first HTTP-level tests. They use Django's test client and a logged-in test user, and they check what a browser or a later request can see. + +Delivered on branch `c1-failure-inputs` as part of the single C1 pull request. The pull request stays open until the release (D5). + +Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`), Change C1 in `docs/failure_roadmap.md`. The Spec issue does not exist yet. GitHub issue #21. + +**Blocked by:** None (can start immediately) + +**Status:** ready-for-agent + +- [ ] The minimum ≤ maximum LRV check is defined once and used by both treatment step forms. Nothing changes that a user can notice, and the existing test suite passes unchanged. +- [ ] A schema migration adds failure frequency (a decimal number) and failure duration (a whole number) to the treatment steps of an assessment. Existing rows get failure frequency 0 and failure duration 30. +- [ ] Each treatment step card in the configurator shows both fields below its LRV rows, labelled "Failure frequency (days per year)" and "Failure duration (minutes)". New steps start at 0 and 30, bundled steps included. +- [ ] Failure frequency outside 0–365, and failure duration outside 1–1440 or not a whole number, are rejected with a message on the saved configurator. The duration range is checked even when the frequency is 0. +- [ ] A fractional failure frequency such as 0.5 is accepted. +- [ ] Saving an assessment and reopening it gives back the same failure values for each step. Editing them and saving again stores the new values. +- [ ] Each value stays with its own step when other steps are removed. Two copies of the same bundled step can have different values. +- [ ] An assessment created without failure values, as before the migration, reopens with failure frequency 0 and failure duration 30. +- [ ] An assessment's result and export package are the same with and without failure inputs. +- [ ] All of the above is covered by HTTP-level tests through Django's test client, and the existing test suite passes unchanged. diff --git a/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md b/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md new file mode 100644 index 0000000..e2abd8e --- /dev/null +++ b/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md @@ -0,0 +1,20 @@ +# 05: Failure frequency only on steps with a positive LRV (D4), with messages for guests too + +**What to build:** A treatment step can only have a failure frequency above 0 if at least one of its LRVs is positive, because a failure removes only positive LRVs (D4). The configurator checks this when the assessment is run or saved, and the error appears on the failure frequency field. + +Guests can use the failure fields too, and the same rules apply to them. Today, when a guest runs an assessment with invalid input, the guest result sends back an empty error and nothing appears on the page. With this Ticket, the guest result sends back the error messages and the configurator shows them on the affected treatment step, as it does for registered users. This also makes the existing LRV errors, such as minimum above maximum, visible to guests. + +Delivered on branch `c1-failure-inputs` as part of the single C1 pull request. + +Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`), roadmap decision D4. The Spec issue does not exist yet. GitHub issue #22. + +**Blocked by:** 04 (#21) + +**Status:** ready-for-agent + +- [ ] A failure frequency above 0 is rejected, with a message on the failure frequency field, if none of the step's six LRVs (minimum and maximum for bacteria, viruses and protozoa) is above 0. An empty LRV counts as 0. +- [ ] A step whose only positive LRV is one maximum (for example protozoa 0–2) accepts a failure frequency above 0. +- [ ] A step with only negative LRVs (recontamination) or only empty LRVs keeps failure frequency 0. A higher value is rejected. +- [ ] A guest sees the failure fields, can run an assessment with valid failure inputs, and nothing is stored. +- [ ] When a guest's input is invalid, whether a failure input or an LRV, the guest result sends back the error messages and the configurator shows them on the affected step. The rules are the same as for registered users. +- [ ] HTTP-level tests cover the D4 cases on the saved configurator and on the guest result, and cover rejection with a message on the guest result. The existing test suite passes unchanged. diff --git a/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md b/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md new file mode 100644 index 0000000..60ecebb --- /dev/null +++ b/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md @@ -0,0 +1,24 @@ +# 06: Personal treatment steps store failure inputs and the configurator copies them in + +**What to build:** A registered user can store a failure frequency and a failure duration on a personal treatment step, with the same rules as in the configurator. When they add the step to an assessment, its failure values are copied in together with its LRVs. The copy can then be changed in that one assessment without changing the personal treatment step. + +Today the personal treatment step form sends back an empty error for invalid input, so the user sees nothing. With this Ticket it shows the messages, the same way it already does for a duplicate name. + +Delivered on branch `c1-failure-inputs` and closes the C1 pull request. The pull request is tested on the dev environment and not merged before the release (D5). + +Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`). The Spec issue does not exist yet. GitHub issue #23. + +**Blocked by:** 04 (#21), 05 (#22) + +**Status:** ready-for-agent + +- [ ] A schema migration adds failure frequency and failure duration to personal treatment steps. Existing personal treatment steps get failure frequency 0 and failure duration 30. +- [ ] The personal treatment step form shows both fields with their units and applies the same shared rule as the configurator: the ranges, whole minutes, D4 and minimum ≤ maximum LRV. Invalid input is rejected and the message is shown in the form. +- [ ] A personal treatment step saved with failure inputs appears with them in its owner's list of personal treatment steps. Another user's list does not contain it. +- [ ] Adding a personal treatment step to an assessment copies its failure frequency and failure duration into the new step. Bundled steps still start at 0 and 30. +- [ ] Changing the copied values in an assessment and saving leaves the personal treatment step unchanged. +- [ ] HTTP-level tests cover the form rules, old personal treatment steps reading back 0 and 30, and owner-only visibility. The existing test suite passes unchanged. +- [ ] Manual check on the dev environment, announced to the team first: + - Create a personal treatment step with failure inputs and add it to an assessment. + - See the copied values, change them, save and reopen. + - Confirm that the personal treatment step itself is unchanged. From 90780b3e3cdc29690c2ef01b85409711ab0be27c Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 25 Sep 2026 10:01:06 +0000 Subject: [PATCH 02/43] docs: add C2 tickets 07-09 (#24, #25, #26); combined-failure arrangement per pathogen [skip ci] The best-case arrangement of combined failure events longer than a day is chosen separately for each reference pathogen; failure days stay shared. Recorded in the C2 Spec and in CONTEXT.md (Combined failure). Co-Authored-By: Claude Opus 5.5 (1M context) --- CONTEXT.md | 2 +- docs/specs/c2-failure-calculation.md | 8 +++-- .../07-c2-calculation-per-exposure-event.md | 18 +++++++++++ .../08-c2-failure-days-and-worst-case.md | 21 +++++++++++++ docs/tickets/09-c2-best-case-mixed-water.md | 30 +++++++++++++++++++ 5 files changed, 76 insertions(+), 3 deletions(-) create mode 100644 docs/tickets/07-c2-calculation-per-exposure-event.md create mode 100644 docs/tickets/08-c2-failure-days-and-worst-case.md create mode 100644 docs/tickets/09-c2-best-case-mixed-water.md diff --git a/CONTEXT.md b/CONTEXT.md index f44d7be..f77a33e 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -175,7 +175,7 @@ The best-case assumption that water consumed on a failure day is a mixture of wa _Avoid_: mixed concentration, dilution, storage effect **Combined failure**: -Failure events in two or more treatment steps on the same day. Worst-case loses all their LRVs for the whole day. Under the mixed-water assumption the failure events are arranged within the day so that the mixed water has the lowest pathogen concentration. This means they do not overlap if their durations fit into one day, and otherwise overlap only by the minutes that exceed a day, shared by the steps whose joint loss matters least. +Failure events in two or more treatment steps on the same day. Worst-case loses all their LRVs for the whole day. Under the mixed-water assumption the failure events are arranged within the day so that the mixed water has the lowest pathogen concentration. This means they do not overlap if their durations fit into one day, and otherwise overlap only by the minutes that exceed a day, shared by the steps whose joint loss matters least. The arrangement is chosen separately for each reference pathogen, since which joint loss matters least depends on the LRVs of its pathogen group. **Next phase**: Capabilities identified for future development and not present in the application today. diff --git a/docs/specs/c2-failure-calculation.md b/docs/specs/c2-failure-calculation.md index 093203e..59e8da9 100644 --- a/docs/specs/c2-failure-calculation.md +++ b/docs/specs/c2-failure-calculation.md @@ -116,7 +116,9 @@ failure frequency is 0 gives exactly the same results as before C2. train minus the removable maximum LRVs of the steps down in it. The failure events of the day are arranged within 1,440 minutes so that this average concentration is lowest. This gives no overlap when the durations fit into one day and otherwise the minimal overlap, - placed on the steps whose joint loss matters least (`CONTEXT.md`, *Combined failure*). For + placed on the steps whose joint loss matters least (`CONTEXT.md`, *Combined failure*). + Because the LRVs differ per pathogen group, the arrangement is chosen separately for each + reference pathogen; the failure days themselves stay shared. For a single failing step it equals Eq. 5 of the failure approach. With the few failing steps per day that occur in practice, the arrangement can be found exactly, for example by choosing the time share of each combination of failing steps. @@ -163,7 +165,9 @@ failure frequency is 0 gives exactly the same results as before C2. by the constant mixed LRV from Eq. 5, computed by hand in the test. - **Combined failures:** two steps always failing with 900 + 900 minutes equal the hand-computed constant LRV with 360 minutes of overlap; three steps always failing with - durations above a day in total equal the hand-computed lowest-concentration arrangement. + durations above a day in total equal the hand-computed lowest-concentration arrangement; + with three steps whose least important pair differs between two pathogen groups, each + reference pathogen equals its own hand-computed arrangement. - **Monotonicity:** raising a failure frequency never lowers any mean; raising a failure duration never lowers a best-case mean and leaves worst-case unchanged. - **Case order:** worst-case mean ≥ best-case mean for every pathogen and risk measure, diff --git a/docs/tickets/07-c2-calculation-per-exposure-event.md b/docs/tickets/07-c2-calculation-per-exposure-event.md new file mode 100644 index 0000000..5dc714c --- /dev/null +++ b/docs/tickets/07-c2-calculation-per-exposure-event.md @@ -0,0 +1,18 @@ +# 07: Tidy-up: the calculation works per exposure event, with results unchanged + +**What to build:** Tidy-up before failures enter the calculation. No result changes. + +The Monte Carlo simulation is reorganised to work per simulated exposure event. Each event picks its concentration sample by position instead of by value, which uses exactly the same random numbers as today, and each event gets its own LRV. For now every event still gets the same LRV. Best-case and worst-case use the same building blocks, and the summary statistics (mean, minimum, quartiles, median, maximum) and the reference-level exceedance are computed in one place for both cases and both risk measures. The calculation's public entry point keeps its interface. + +Delivered on branch `c2-failure-calculation`, based on `c1-failure-inputs`, as part of the single C2 pull request. The pull request stays open until the release (D5). + +Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), Change C2 in `docs/failure_roadmap.md`. The Spec issue does not exist yet. GitHub issue #24. + +**Blocked by:** None (can start immediately) + +**Status:** ready-for-agent + +- [ ] Every simulated exposure event has its own concentration sample, picked by position, and its own LRV. The random numbers are the same as today's. +- [ ] Best-case and worst-case share the same calculation steps. Summary statistics and the reference-level exceedance are computed in one place. +- [ ] The existing regression test passes with exactly the same numbers, and the existing test suite passes unchanged. +- [ ] A new test at the calculation's public entry point shows that calculating the same scenario twice gives identical results. diff --git a/docs/tickets/08-c2-failure-days-and-worst-case.md b/docs/tickets/08-c2-failure-days-and-worst-case.md new file mode 100644 index 0000000..a2e3428 --- /dev/null +++ b/docs/tickets/08-c2-failure-days-and-worst-case.md @@ -0,0 +1,21 @@ +# 08: Failure days drawn per exposure event; worst-case loses the positive LRVs of failing steps + +**What to build:** The worst-case result includes treatment failures. For every simulated exposure event, and for every treatment step with a failure frequency above 0, the step fails that day with probability failure frequency / 365 (ADR-0001). On a failure day, worst-case calculates the event with the positive minimum LRVs of every failing step removed, whatever the failure duration. LRVs of 0, below 0 or empty stay in place (D4). + +The failure draws come from their own fixed-seed random generator and are shared by both cases and all reference pathogens. Raising a failure frequency therefore only adds failure days. If every failure frequency is 0, no failure draws are made and the results are exactly today's. Best-case does not change in this Ticket; the mixed-water assumption follows in the next one. + +The result page, the reference-level exceedance, saved results and the export's result table show whatever the calculation returns. Their layout does not change. + +Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), ADR-0001, roadmap decision D4. The Spec issue does not exist yet. GitHub issue #25. + +**Blocked by:** 07 (#24), 04 (#21, C1 stored failure inputs) + +**Status:** ready-for-agent + +- [ ] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. +- [ ] A step with failure frequency 365 gives the same worst-case results as the same scenario without that step's positive minimum LRVs. +- [ ] Raising a failure frequency never lowers any mean. Changing a failure duration leaves worst-case unchanged. +- [ ] Worst-case mean ≥ best-case mean for every reference pathogen and risk measure, with and without failures. +- [ ] D4: a failing step with a negative LRV in one pathogen group and a positive LRV in another leaves the first group's results unchanged. A group with LRV 0 is unaffected. +- [ ] Calculating the same scenario with failures twice gives identical results. +- [ ] All tests go through the calculation's public entry point and check only the returned results. The existing test suite passes unchanged. diff --git a/docs/tickets/09-c2-best-case-mixed-water.md b/docs/tickets/09-c2-best-case-mixed-water.md new file mode 100644 index 0000000..1485cd4 --- /dev/null +++ b/docs/tickets/09-c2-best-case-mixed-water.md @@ -0,0 +1,30 @@ +# 09: Best-case applies the mixed-water assumption, including combined failures + +**What to build:** The best-case result includes treatment failures under the mixed-water assumption. On a failure day, the consumed water is a mixture of water treated during the failure events and water treated in normal operation, in proportion to the failure durations. The event's LRV is −log₁₀ of the time-weighted average of 10^(−LRV) over the segments of the day, where a segment's LRV is the best-case LRV of the train minus the positive maximum LRVs of the steps down in that segment. + +The failure events of the day are arranged within 1,440 minutes so that the mixed concentration is lowest: +- if their durations fit into one day, they do not overlap; +- otherwise they overlap only as much as necessary, on the steps whose joint loss matters least. + +The arrangement is chosen separately for each reference pathogen, because the LRVs differ per pathogen group. The failure days themselves stay shared. For a single failing step, the result equals Eq. 5 of the failure approach. Work per failure day is reused for repeated combinations of failing steps. + +This Ticket closes C2: it includes the manual check on the dev environment and the time measurement. + +Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), ADR-0001, `CONTEXT.md` (*Mixed-water assumption*, *Combined failure*), decision 6 in `docs/QMRA_Failure_Approach.md`. The Spec issue does not exist yet. GitHub issue #26. + +**Blocked by:** 08 (#25) + +**Status:** ready-for-agent + +- [ ] A step with failure frequency 365 and failure duration 1,440 gives the same best-case results as the same scenario without that step's positive maximum LRVs. +- [ ] A step with failure frequency 365 and failure duration 60 gives the same best-case results as the scenario with the train's best-case LRV replaced by the Eq. 5 mixed LRV, computed by hand in the test. +- [ ] Two steps always failing with 900 + 900 minutes equal the hand-computed result with 360 minutes of overlap. +- [ ] Three steps always failing with more than a day in total equal the hand-computed lowest-concentration arrangement. +- [ ] Three steps whose least important pair differs between two pathogen groups: each reference pathogen equals its own hand-computed arrangement. +- [ ] Raising a failure duration never lowers a best-case mean, and raising a failure frequency never lowers any mean. +- [ ] Worst-case mean ≥ best-case mean for every reference pathogen and risk measure, with failures. +- [ ] With every failure frequency at 0, all results equal today's exactly. The same scenario gives identical results every time. The existing test suite passes unchanged. +- [ ] Manual check on the dev environment, announced to the team first: + - Enter a scenario with failures and run it as a guest and as a registered user. + - Confirm that the result page, the saved-assessments page, the assessment comparison and the export's result table show the changed results, with unchanged layout. +- [ ] The calculation time for a drinking-water scenario (365 events per year) with failures is measured on the dev environment and recorded in the pull request. From 57bdd218a4a19d73c2b64e2c032d66da2205def6 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 25 Sep 2026 10:15:42 +0000 Subject: [PATCH 03/43] feat: failure inputs on treatment steps in the configurator (#21) Each treatment step of an assessment gets a failure frequency (failure days per year, 0-365, default 0) and a failure duration (whole minutes, 1-1440, default 30). The configurator shows both fields with their units below the LRVs, rejects values outside the ranges with a message, and keeps the values with their step through save, reopen and edit. Existing steps get the defaults via the migration. The calculation is unchanged, so results and export packages stay the same (C2 adds the effect). Tidy-up: the min <= max LRV check duplicated in the configurator and the personal treatment step form is now one shared rule, which also holds the failure ranges. The two existing treatment form tests pass the two new required fields in their input data; their assertions are unchanged. Adds the first HTTP-level tests (Django test client, logged-in user). Refs #21 Co-Authored-By: Claude Opus 5.5 (1M context) --- qmra/risk_assessment/forms.py | 49 ++-- .../0011_treatment_failure_inputs.py | 23 ++ qmra/risk_assessment/models.py | 2 + .../tests/test_failure_inputs.py | 232 ++++++++++++++++++ .../tests/test_risk_assessment_form.py | 8 +- qmra/risk_assessment/user_models.py | 16 +- 6 files changed, 300 insertions(+), 30 deletions(-) create mode 100644 qmra/risk_assessment/migrations/0011_treatment_failure_inputs.py create mode 100644 qmra/risk_assessment/tests/test_failure_inputs.py diff --git a/qmra/risk_assessment/forms.py b/qmra/risk_assessment/forms.py index 57b1bc7..ce589e7 100644 --- a/qmra/risk_assessment/forms.py +++ b/qmra/risk_assessment/forms.py @@ -14,6 +14,32 @@ def _zero_if_none(x): return x if x is not None else 0 +LRV_FIELDS = [ + ("bacteria_min", "bacteria_max"), + ("viruses_min", "viruses_max"), + ("protozoa_min", "protozoa_max"), +] +FAILURE_INPUT_RANGES = { + "failure_frequency": (0, 365, "days per year"), + "failure_duration": (1, 1440, "minutes"), +} + + +def check_treatment_step(form, cleaned_data): + """Minimum LRV at most maximum LRV per pathogen group, and failure inputs within their ranges. + + Shared by the configurator and the personal treatment step form. Whole minutes are enforced + by the failure duration's field type; failure inputs a form does not have are skipped. + """ + for min_field, max_field in LRV_FIELDS: + if _zero_if_none(cleaned_data.get(min_field, 0)) > _zero_if_none(cleaned_data.get(max_field, 0)): + form.add_error(min_field, "min. must be less than max") + for field, (low, high, unit) in FAILURE_INPUT_RANGES.items(): + value = cleaned_data.get(field) + if value is not None and not low <= value <= high: + form.add_error(field, f"{field.replace('_', ' ')} must be between {low} and {high} {unit}") + + class RiskAssessmentForm(forms.ModelForm): source_name = forms.ChoiceField() exposure_name = forms.ChoiceField() @@ -150,7 +176,9 @@ class Meta: 'viruses_min', 'viruses_max', "protozoa_min", - "protozoa_max" + "protozoa_max", + "failure_frequency", + "failure_duration", ] def __init__(self, *args, **kwargs): @@ -167,6 +195,10 @@ def __init__(self, *args, **kwargs): self.fields['viruses_max'].label = "" self.fields['protozoa_min'].label = "" self.fields['protozoa_max'].label = "" + self.fields['failure_frequency'].label = "Failure frequency (days per year)" + self.fields['failure_duration'].label = "Failure duration (minutes)" + # leave whole minutes to the server check: the browser blocks hidden step cards without a message + self.fields['failure_duration'].widget.attrs['step'] = "any" label_style = "class='text-muted text-center w-100' style='margin-top: .4em;'" self.helper.layout = Layout( Field("name", css_class="disabled-input d-none"), @@ -179,24 +211,13 @@ def __init__(self, *args, **kwargs): Column("viruses_min"), Column("viruses_max")), Row(Column(HTML(f"")), Column("protozoa_min"), Column("protozoa_max")), + Row(Column("failure_frequency"), Column("failure_duration")), # Row(Column("DELETE")) ) def clean(self): cleaned_data = super().clean() - b_min = _zero_if_none(cleaned_data.get("bacteria_min", 0)) - b_max = _zero_if_none(cleaned_data.get("bacteria_max", 0)) - v_min = _zero_if_none(cleaned_data.get("viruses_min", 0)) - v_max = _zero_if_none(cleaned_data.get("viruses_max", 0)) - p_min = _zero_if_none(cleaned_data.get("protozoa_min", 0)) - p_max = _zero_if_none(cleaned_data.get("protozoa_max", 0)) - msg = "min. must be less than max" - if b_min > b_max: - self.add_error("bacteria_min", msg) - if v_min > v_max: - self.add_error("viruses_min", msg) - if p_min > p_max: - self.add_error("protozoa_min", msg) + check_treatment_step(self, cleaned_data) return cleaned_data diff --git a/qmra/risk_assessment/migrations/0011_treatment_failure_inputs.py b/qmra/risk_assessment/migrations/0011_treatment_failure_inputs.py new file mode 100644 index 0000000..b05f235 --- /dev/null +++ b/qmra/risk_assessment/migrations/0011_treatment_failure_inputs.py @@ -0,0 +1,23 @@ +# Generated by Django 5.0.14 on 2026-09-25 10:10 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('risk_assessment', '0010_remove_qmratreatment_bacteria_reference_and_more'), + ] + + operations = [ + migrations.AddField( + model_name='treatment', + name='failure_duration', + field=models.IntegerField(default=30), + ), + migrations.AddField( + model_name='treatment', + name='failure_frequency', + field=models.FloatField(default=0), + ), + ] diff --git a/qmra/risk_assessment/models.py b/qmra/risk_assessment/models.py index 030def7..eaeb936 100644 --- a/qmra/risk_assessment/models.py +++ b/qmra/risk_assessment/models.py @@ -35,6 +35,8 @@ class Treatment(models.Model): viruses_max = models.FloatField(blank=True, null=True) protozoa_min = models.FloatField(blank=True, null=True) protozoa_max = models.FloatField(blank=True, null=True) + failure_frequency = models.FloatField(default=0) # failure days per year + failure_duration = models.IntegerField(default=30) # minutes @classmethod def from_default(cls, default: QMRATreatment, risk_assessment): diff --git a/qmra/risk_assessment/tests/test_failure_inputs.py b/qmra/risk_assessment/tests/test_failure_inputs.py new file mode 100644 index 0000000..ffc038a --- /dev/null +++ b/qmra/risk_assessment/tests/test_failure_inputs.py @@ -0,0 +1,232 @@ +"""failure inputs of treatment steps, tested over HTTP like a browser uses the configurator""" +import io +from html.parser import HTMLParser +from zipfile import ZipFile + +from assertpy import assert_that +from django.test import TestCase +from django.urls import reverse + +from qmra.risk_assessment.models import RiskAssessment, Inflow, Treatment +from qmra.user.models import User + + +class ConfiguratorFormParser(HTMLParser): + """Collects what a browser would submit from the configurator form.""" + + def __init__(self): + super().__init__() + self.data = {} + self.in_configurator = False + self.select_name = None + self.first_option = None + + def handle_starttag(self, tag, attrs): + attrs = dict(attrs) + if tag == "form": + self.in_configurator = attrs.get("id") == "configurator" + if not self.in_configurator: + return + if tag == "input" and attrs.get("name"): + if attrs.get("type") in ("submit", "button"): + return + if attrs.get("type") == "checkbox" and "checked" not in attrs: + return + self.data[attrs["name"]] = attrs.get("value", "") + elif tag == "select": + self.select_name = attrs.get("name") + self.first_option = None + elif tag == "option" and self.select_name: + if self.first_option is None: + self.first_option = attrs.get("value", "") + self.data.setdefault(self.select_name, self.first_option) + if "selected" in attrs: + self.data[self.select_name] = attrs.get("value", "") + + def handle_endtag(self, tag): + if tag == "form": + self.in_configurator = False + elif tag == "select": + self.select_name = None + + +def configurator_data(response) -> dict: + parser = ConfiguratorFormParser() + parser.feed(response.content.decode()) + return parser.data + + +def add_step(data: dict, name: str, **values): + """Add a treatment step card the way the configurator's script does: copy the empty form.""" + n = int(data["treatments-TOTAL_FORMS"]) + for key, value in list(data.items()): + if key.startswith("treatments-__prefix__-") and not key.endswith("-DELETE"): + data[key.replace("__prefix__", str(n))] = value + data[f"treatments-{n}-name"] = name + for field, value in values.items(): + data[f"treatments-{n}-{field}"] = value + data["treatments-TOTAL_FORMS"] = str(n + 1) + + +def failure_inputs_per_step(data: dict) -> list[dict]: + """The treatment step cards of a configurator, in order, with their failure inputs.""" + return [ + dict(name=data[f"treatments-{i}-name"], + failure_frequency=float(data[f"treatments-{i}-failure_frequency"]), + failure_duration=int(data[f"treatments-{i}-failure_duration"])) + for i in range(int(data["treatments-TOTAL_FORMS"])) + ] + + +LRVS = dict(bacteria_min=1, bacteria_max=2, viruses_min=1, viruses_max=2, protozoa_min=1, protozoa_max=2) + + +class TestFailureInputs(TestCase): + + def setUp(self): + self.user = User.objects.create_user("test-user", "test-user@test.com", "password") + self.client.force_login(self.user) + + def new_configurator(self) -> dict: + data = configurator_data(self.client.get(reverse("assessment"))) + for i in range(3): + data[f"inflow-{i}-min"] = "10" + data[f"inflow-{i}-max"] = "100" + data["ra-source_name"] = "groundwater" + data["ra-exposure_name"] = "drinking water" + data["ra-events_per_year"] = "365" + data["ra-volume_per_event"] = "1" + return data + + def post_configurator(self, data: dict, assessment: RiskAssessment = None): + url = reverse("assessment") if assessment is None else reverse("assessment", args=[assessment.id]) + return self.client.post(url, data) + + def save_new(self, data: dict) -> RiskAssessment: + response = self.post_configurator(data) + assert_that(response.status_code).is_equal_to(302) + return RiskAssessment.objects.get(user=self.user) + + def reopen(self, assessment: RiskAssessment) -> dict: + return configurator_data(self.client.get(reverse("assessment", args=[assessment.id]))) + + def results(self, assessment: RiskAssessment) -> dict: + return {r.pathogen: r.as_dict() for r in RiskAssessment.objects.get(id=assessment.id).results.all()} + + def export(self, assessment: RiskAssessment) -> dict: + response = self.client.get(reverse("assessment-export", args=[assessment.id])) + with ZipFile(io.BytesIO(response.content)) as archive: + return {name: archive.read(name) for name in archive.namelist()} + + def test_new_step_card_shows_failure_fields_with_units_and_defaults(self): + response = self.client.get(reverse("assessment")) + + assert_that(response.content.decode()).contains( + "Failure frequency (days per year)", "Failure duration (minutes)") + data = configurator_data(response) + assert_that(float(data["treatments-__prefix__-failure_frequency"])).is_equal_to(0) + assert_that(int(data["treatments-__prefix__-failure_duration"])).is_equal_to(30) + + def test_bundled_step_starts_at_defaults(self): + data = self.new_configurator() + add_step(data, "Primary treatment", **LRVS) + + assessment = self.save_new(data) + + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Primary treatment", failure_frequency=0, failure_duration=30) + ]) + + def test_failure_inputs_survive_save_and_reopen_per_step(self): + data = self.new_configurator() + add_step(data, "Primary treatment", **LRVS, failure_frequency="0.5", failure_duration="15") + add_step(data, "Primary treatment", **LRVS, failure_frequency="12", failure_duration="600") + add_step(data, "Slow sand filtration", **LRVS, failure_frequency="365", failure_duration="1440") + + assessment = self.save_new(data) + + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Primary treatment", failure_frequency=0.5, failure_duration=15), + dict(name="Primary treatment", failure_frequency=12, failure_duration=600), + dict(name="Slow sand filtration", failure_frequency=365, failure_duration=1440), + ]) + + def test_editing_stores_new_values_and_values_stay_with_their_step(self): + data = self.new_configurator() + add_step(data, "Primary treatment", **LRVS, failure_frequency="1", failure_duration="10") + add_step(data, "Slow sand filtration", **LRVS, failure_frequency="2", failure_duration="20") + add_step(data, "Primary treatment", **LRVS, failure_frequency="3", failure_duration="30") + assessment = self.save_new(data) + + data = self.reopen(assessment) + data["treatments-0-DELETE"] = "on" + data["treatments-2-failure_frequency"] = "4.5" + data["treatments-2-failure_duration"] = "45" + response = self.post_configurator(data, assessment) + + assert_that(response.status_code).is_equal_to(302) + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Slow sand filtration", failure_frequency=2, failure_duration=20), + dict(name="Primary treatment", failure_frequency=4.5, failure_duration=45), + ]) + + def test_out_of_range_or_fractional_values_are_rejected_with_a_message(self): + cases = [ + ("-1", "30", "failure frequency must be between 0 and 365 days per year"), + ("365.5", "30", "failure frequency must be between 0 and 365 days per year"), + ("0", "0", "failure duration must be between 1 and 1440 minutes"), + ("0", "1441", "failure duration must be between 1 and 1440 minutes"), + ("1", "2.5", "Enter a whole number."), + ("1", "", "This field is required."), + ("", "30", "This field is required."), + ] + for frequency, duration, message in cases: + with self.subTest(frequency=frequency, duration=duration): + data = self.new_configurator() + add_step(data, "Primary treatment", **LRVS, + failure_frequency=frequency, failure_duration=duration) + + response = self.post_configurator(data) + + assert_that(response.status_code).is_equal_to(200) + assert_that(response.content.decode()).contains(message) + assert_that(RiskAssessment.objects.filter(user=self.user).count()).is_equal_to(0) + + def test_range_bounds_are_accepted(self): + for frequency, duration in [("0", "1"), ("365", "1440"), ("0.5", "30")]: + with self.subTest(frequency=frequency, duration=duration): + data = self.new_configurator() + add_step(data, "Primary treatment", **LRVS, + failure_frequency=frequency, failure_duration=duration) + + response = self.post_configurator(data) + + assert_that(response.status_code).is_equal_to(302) + + def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): + assessment = RiskAssessment.objects.create(user=self.user, events_per_year=365, volume_per_event=1) + for pathogen in ["Rotavirus", "Campylobacter jejuni", "Cryptosporidium parvum"]: + Inflow.objects.create(risk_assessment=assessment, pathogen=pathogen, min=10, max=100) + Treatment.objects.create(risk_assessment=assessment, name="Primary treatment", **LRVS) + + assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ + dict(name="Primary treatment", failure_frequency=0, failure_duration=30) + ]) + + def test_result_and_export_do_not_change_with_failure_inputs(self): + data = self.new_configurator() + add_step(data, "Primary treatment", **LRVS) + add_step(data, "Slow sand filtration", **LRVS) + assessment = self.save_new(data) + results_before = self.results(assessment) + export_before = self.export(assessment) + + data = self.reopen(assessment) + data["treatments-0-failure_frequency"] = "20" + data["treatments-0-failure_duration"] = "120" + data["treatments-1-failure_frequency"] = "365" + self.post_configurator(data, assessment) + + assert_that(failure_inputs_per_step(self.reopen(assessment))[0]["failure_frequency"]).is_equal_to(20) + assert_that(self.results(assessment)).is_equal_to(results_before) + assert_that(self.export(assessment)).is_equal_to(export_before) diff --git a/qmra/risk_assessment/tests/test_risk_assessment_form.py b/qmra/risk_assessment/tests/test_risk_assessment_form.py index e28527d..d1ae84b 100644 --- a/qmra/risk_assessment/tests/test_risk_assessment_form.py +++ b/qmra/risk_assessment/tests/test_risk_assessment_form.py @@ -87,7 +87,9 @@ def test_that_negative_are_allowed(self): viruses_min=-2, viruses_max=-1, protozoa_min=-2, - protozoa_max=-1 + protozoa_max=-1, + failure_frequency=0, + failure_duration=30, ) given_form = TreatmentForm(data=data) # ugly hack to work around dynamic choices... @@ -104,7 +106,9 @@ def test_that_min_needs_to_be_less_than_max(self): viruses_min=0, viruses_max=0, protozoa_min=0, - protozoa_max=0 + protozoa_max=0, + failure_frequency=0, + failure_duration=30, ) for prfx in ["bacteria", "viruses", "protozoa"]: mn = prfx + "_min" diff --git a/qmra/risk_assessment/user_models.py b/qmra/risk_assessment/user_models.py index 3b5f481..6f356c3 100644 --- a/qmra/risk_assessment/user_models.py +++ b/qmra/risk_assessment/user_models.py @@ -6,7 +6,7 @@ from crispy_forms.helper import FormHelper from crispy_forms.layout import Layout, Field, Row, Column, HTML, Submit -from qmra.risk_assessment.forms import _zero_if_none +from qmra.risk_assessment.forms import check_treatment_step from qmra.user.models import User @@ -184,17 +184,5 @@ def __init__(self, *args, **kwargs): def clean(self): cleaned_data = super().clean() - b_min = _zero_if_none(cleaned_data.get("bacteria_min", 0)) - b_max = _zero_if_none(cleaned_data.get("bacteria_max", 0)) - v_min = _zero_if_none(cleaned_data.get("viruses_min", 0)) - v_max = _zero_if_none(cleaned_data.get("viruses_max", 0)) - p_min = _zero_if_none(cleaned_data.get("protozoa_min", 0)) - p_max = _zero_if_none(cleaned_data.get("protozoa_max", 0)) - msg = "min. must be less than max" - if b_min > b_max: - self.add_error("bacteria_min", msg) - if v_min > v_max: - self.add_error("viruses_min", msg) - if p_min > p_max: - self.add_error("protozoa_min", msg) + check_treatment_step(self, cleaned_data) return cleaned_data From 52e608788572594801104012f1a1ec5c94f6f0c7 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 25 Sep 2026 10:29:37 +0000 Subject: [PATCH 04/43] feat: failure only on steps with a positive LRV, and messages for guests (#22) A treatment step can have a failure frequency above 0 only if at least one of its six LRVs is positive (an empty LRV counts as 0), since a failure removes only positive LRVs (D4). The shared treatment step rule rejects it with a message on the failure frequency field. Guests: an invalid guest result now answers 422 with the error messages keyed by input name instead of an empty body. The configurator marks the affected inputs and treatment step and explains in the result pane why nothing ran; the marks clear on the next run. Registered users keep their path (the saved configurator re-renders with the errors). This also makes existing LRV errors visible to guests. Also fixes the treatments section from #21: its fixed height hid the failure fields and their messages under the footer; it now grows with its content. Refs #22 Co-Authored-By: Claude Opus 5.5 (1M context) --- qmra/risk_assessment/forms.py | 7 +- .../templates/risk-assessment-form-js.html | 57 ++++++- .../templates/treatments-form-fieldset.html | 2 +- .../tests/test_failure_inputs.py | 151 ++++++++++++++++-- qmra/risk_assessment/views.py | 24 ++- 5 files changed, 218 insertions(+), 23 deletions(-) diff --git a/qmra/risk_assessment/forms.py b/qmra/risk_assessment/forms.py index ce589e7..d08f4df 100644 --- a/qmra/risk_assessment/forms.py +++ b/qmra/risk_assessment/forms.py @@ -26,7 +26,8 @@ def _zero_if_none(x): return x if x is not None else 0 def check_treatment_step(form, cleaned_data): - """Minimum LRV at most maximum LRV per pathogen group, and failure inputs within their ranges. + """Minimum LRV at most maximum LRV per pathogen group, failure inputs within their ranges, + and a failure frequency above 0 only for a step with a positive LRV. Shared by the configurator and the personal treatment step form. Whole minutes are enforced by the failure duration's field type; failure inputs a form does not have are skipped. @@ -38,6 +39,10 @@ def check_treatment_step(form, cleaned_data): value = cleaned_data.get(field) if value is not None and not low <= value <= high: form.add_error(field, f"{field.replace('_', ' ')} must be between {low} and {high} {unit}") + # a failure removes only positive LRVs, so a step without one cannot fail (D4) + has_positive_lrv = any(_zero_if_none(cleaned_data.get(field)) > 0 for pair in LRV_FIELDS for field in pair) + if _zero_if_none(cleaned_data.get("failure_frequency")) > 0 and not has_positive_lrv: + form.add_error("failure_frequency", "failure frequency must be 0 for a treatment step without a positive LRV") class RiskAssessmentForm(forms.ModelForm): diff --git a/qmra/risk_assessment/templates/risk-assessment-form-js.html b/qmra/risk_assessment/templates/risk-assessment-form-js.html index c414676..403bc5f 100644 --- a/qmra/risk_assessment/templates/risk-assessment-form-js.html +++ b/qmra/risk_assessment/templates/risk-assessment-form-js.html @@ -1,6 +1,57 @@ {% load static %} + + + + + + +
+
+

Result

+ +
+
The risk of this water reuse scenario can be compared to + two commonly applied health-based targets (WHO 2022): +
+
    +
  1. 1 in 10,000 infections per person per year
    2. +
  2. 0.000001 Disability Adjusted Life Years (DALY)
    3. +
+ +
+ The estimated probability of infection per year / DALYs pppy + from one or more pathogens exceeds the tolerable risk level + indicated by the WHO calculated using the minimum LRV. +
+
+ A closer look into the individual treatments + is necessary to evaluate the risk of the scenario. + +
+ +
+ + + +
+ +
+ + +
+ +
+ + + +
+ + +
+ + + \ No newline at end of file diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv new file mode 100644 index 0000000..36ceeb2 --- /dev/null +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv @@ -0,0 +1,12 @@ +"Risk Assessment's Name:","Reference assessment" +"Description:","Benchmark for the export package" + +"Result's Summary:" +,,Infection prob.,Infection prob.,Infection prob.,Infection prob.,Infection prob.,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy +pathogen,stat,min,25%,50%,75%,max,min,25%,50%,75%,max +Rotavirus,Maximum LRV,9.572485276665432e-07,1.4836628104142324e-06,1.7097025438728153e-06,1.931930578930041e-06,2.893425146477391e-06,6.700739693665803e-09,1.0385639672899628e-08,1.1967917807109707e-08,1.3523514052510289e-08,2.0253976025341736e-08 +Rotavirus,Minimum LRV,0.3640096391082689,0.41122481268750266,0.4267809590038193,0.44364273411319005,0.5027473445430916,0.002548067473757882,0.002878573688812519,0.002987466713026735,0.0031054991387923303,0.003519231411801641 +Campylobacter jejuni,Maximum LRV,4.5133408121955654e-10,2.2749717942804892e-09,3.0147686569392818e-09,3.744822868467779e-09,7.008461921920173e-09,6.22841032082988e-13,3.139461076107075e-12,4.160380746576208e-12,5.167855558485534e-12,9.671677452249837e-12 +Campylobacter jejuni,Minimum LRV,0.0001083267586116099,0.0005457054173893527,0.0007230268318788591,0.0008980393643011064,0.0016799226351181806,1.4949092688402165e-07,7.530734759973068e-07,9.977770279928255e-07,1.2392943227355267e-06,2.3182932364630894e-06 +Cryptosporidium parvum,Maximum LRV,9.220223140538053e-09,1.429064969138416e-08,1.6467869567993176e-08,1.860837314793251e-08,2.7869510144107323e-08,9.681234297564956e-12,1.5005182175953368e-11,1.7291263046392836e-11,1.9538791805329135e-11,2.926298565131269e-11 +Cryptosporidium parvum,Minimum LRV,0.0028129655229737205,0.009066386874888782,0.011592701982251985,0.01415027039915931,0.025193602188789632,2.9536137991224066e-06,9.51970621863322e-06,1.2172337081364584e-05,1.4857783919117274e-05,2.645328229822911e-05 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv new file mode 100644 index 0000000..d2a8fbf --- /dev/null +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv @@ -0,0 +1,4 @@ +Pathogen,Minimum Concentration,Maximum Concentration +Rotavirus,0.01,0.1 +Campylobacter jejuni,0.1,1.0 +Cryptosporidium parvum,0.01,0.1 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv new file mode 100644 index 0000000..428fc08 --- /dev/null +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv @@ -0,0 +1,7 @@ +Treatment,Pathogen group,Maximum LRV,Minimum LRV +Slow sand filtration,Viruses,4.0,0.25 +Slow sand filtration,Bacteria,6.0,2.0 +Slow sand filtration,Protozoa,5.0,0.3 +"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0 +"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6 +"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5 diff --git a/qmra/risk_assessment/tests/test_reference_export.py b/qmra/risk_assessment/tests/test_reference_export.py new file mode 100644 index 0000000..711aebe --- /dev/null +++ b/qmra/risk_assessment/tests/test_reference_export.py @@ -0,0 +1,90 @@ +"""the export package changes only on purpose (quality goal Export stability)""" +import io +import os +import re +import shutil +from pathlib import Path +from zipfile import ZipFile + +from assertpy import assert_that +from django.test import TestCase +from django.urls import reverse + +from qmra.risk_assessment.models import RiskAssessment, Inflow, Treatment +from qmra.user.models import User + +REFERENCE = Path(__file__).parent / "reference_export" +FILE_LIST = REFERENCE / "files.txt" +PACKAGE = REFERENCE / "package" +# the plots are only checked for presence: their rendering can differ between machines +PLOTS = ["results-plots/infection-probability.png", "results-plots/dalys-pppy.png"] +UPDATE_REFERENCE = os.getenv("UPDATE_REFERENCE_EXPORT") == "1" + + +def is_compared(name: str) -> bool: + """Whether a ZIP entry is compared by content: not a folder and not a plot.""" + return not name.endswith("/") and name not in PLOTS + + +def masked(name: str, content: bytes) -> str: + """The content of an exported file as it is compared; the plots embedded in the report are masked.""" + text = content.decode("utf-8") + if name.endswith(".html"): + text, plots = re.subn(r"base64,\s*[A-Za-z0-9+/=]+", "base64, ", text) + assert_that(plots).described_as(f"plots masked in {name}").is_equal_to(len(PLOTS)) + return text + + +class TestReferenceExport(TestCase): + + def setUp(self): + self.user = User.objects.create_user("reference-user", "reference-user@test.com", "password") + self.client.force_login(self.user) + self.assessment = RiskAssessment.objects.create( + user=self.user, name="Reference assessment", description="Benchmark for the export package", + source_name="groundwater", exposure_name="drinking water", events_per_year=365, volume_per_event=1) + # fixed values rather than bundled data, so a data release doesn't change the reference + for pathogen, (low, high) in {"Rotavirus": (0.01, 0.1), "Campylobacter jejuni": (0.1, 1), + "Cryptosporidium parvum": (0.01, 0.1)}.items(): + Inflow.objects.create(risk_assessment=self.assessment, pathogen=pathogen, min=low, max=high) + steps = [ + dict(name="Slow sand filtration", bacteria_min=2, bacteria_max=6, viruses_min=0.25, viruses_max=4, + protozoa_min=0.3, protozoa_max=5), + dict(name="UV disinfection 20 mJ/cm2, drinking", bacteria_min=4.6, bacteria_max=6, viruses_min=1, + viruses_max=3, protozoa_min=2.5, protozoa_max=3, failure_frequency=10, failure_duration=60), + ] + for index, step in enumerate(steps): + Treatment.objects.create(risk_assessment=self.assessment, train_index=index, **step) + + def export(self) -> dict[str, bytes]: + response = self.client.get(reverse("assessment-export", args=[self.assessment.id])) + assert_that(response.status_code).is_equal_to(200) + with ZipFile(io.BytesIO(response.content)) as archive: + return {name: archive.read(name) for name in archive.namelist()} + + def update_reference(self, export: dict[str, bytes]): + shutil.rmtree(PACKAGE, ignore_errors=True) + FILE_LIST.parent.mkdir(parents=True, exist_ok=True) + FILE_LIST.write_text("".join(f"{name}\n" for name in sorted(export)), encoding="utf-8") + for name, content in export.items(): + if is_compared(name): + path = PACKAGE / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(masked(name, content), encoding="utf-8") + + def test_export_package_matches_the_reference(self): + export = self.export() + if UPDATE_REFERENCE: + self.update_reference(export) + self.skipTest("reference export updated; review the changes before committing them") + + assert_that(str(FILE_LIST)).described_as("no reference export stored yet").exists() + assert_that(sorted(export)).described_as("files of the export package").is_equal_to( + FILE_LIST.read_text(encoding="utf-8").splitlines()) + for name, content in export.items(): + if name in PLOTS: + assert_that(content).described_as(name).is_not_empty() + elif is_compared(name): + assert_that(str(PACKAGE / name)).described_as(f"reference of {name}").exists() + assert_that(masked(name, content)).described_as(name).is_equal_to( + (PACKAGE / name).read_text(encoding="utf-8")) From a04e2f953351061b7d5a81d5e5162ac4bd0facf9 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 25 Sep 2026 20:34:13 +0000 Subject: [PATCH 14/43] feat: failure inputs in the treatment table of the export (#31) The export package records which failures a result is based on: treatments.csv gets the columns "Failure frequency (days per year)" and "Failure duration (minutes)", with each step's values on all three of its pathogen-group rows, so the file keeps one row per step and group. The HTML report shows no inputs at all and stays so (decided 2026-09-25); the roadmap's C3 entry says so now. The reference export is updated on purpose: only treatments.csv changed. The C1 test for failure frequency 0 now allows the treatment table to show the new durations, while results and every other exported file stay identical. Release note: treatments.csv in the export package has two new columns at the end. Refs #31 Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/failure_roadmap.md | 14 ++++---- qmra/risk_assessment/exports.py | 5 ++- .../exposure-assessment/treatments.csv | 14 ++++---- .../tests/test_failure_inputs.py | 32 +++++++++++++++++-- 4 files changed, 49 insertions(+), 16 deletions(-) diff --git a/docs/failure_roadmap.md b/docs/failure_roadmap.md index 3dda3ef..7c54ebf 100644 --- a/docs/failure_roadmap.md +++ b/docs/failure_roadmap.md @@ -21,7 +21,7 @@ at least one positive LRV accept failure inputs (D4); values of a personal treat carried into the assessment; saving, reopening and editing keep them. Existing saved assessments get failure frequency 0, so their results are unchanged. -Not included: any effect on the calculation (C2), export and report (C3), explanations +Not included: any effect on the calculation (C2), the export (C3), explanations beyond field labels and units (C4). ### C2 Failure calculation — Large @@ -46,12 +46,14 @@ Not included: the comparison with an independent calculation (C5), showing norma operation next to the result with failures (decided against), DALYs in the assessment comparison (later). -### C3 Failure inputs in export and report — Small +### C3 Failure inputs in the export — Small -The export package's treatment table and the HTML report show each treatment step's failure -frequency and failure duration, so a reader can see which failures a result is based on. -Left out: renaming export files (`inflows.csv` → `inflow_concentration.csv` waits until -later), origin and reference columns for inputs, changes to the assessment comparison. +The export package's treatment table shows each treatment step's failure frequency and +failure duration, so a reader can see which failures a result is based on. Left out: the +HTML report, which shows no inputs at all and so doesn't show failure inputs either +(decided 2026-09-25); renaming export files (`inflows.csv` → `inflow_concentration.csv` +waits until later); origin and reference columns for inputs; changes to the assessment +comparison. ### C4 Explain treatment failure — Small diff --git a/qmra/risk_assessment/exports.py b/qmra/risk_assessment/exports.py index f70aa62..f240e9c 100644 --- a/qmra/risk_assessment/exports.py +++ b/qmra/risk_assessment/exports.py @@ -27,7 +27,10 @@ def treatments_as_df(treatments: QuerySet[Treatment]) -> pd.DataFrame: "Treatment": [t.name] * 3, "Pathogen group": ["Viruses", "Bacteria", "Protozoa"], "Maximum LRV": [t.viruses_max, t.bacteria_max, t.protozoa_max], - "Minimum LRV": [t.viruses_min, t.bacteria_min, t.protozoa_min] + "Minimum LRV": [t.viruses_min, t.bacteria_min, t.protozoa_min], + # a step's failure inputs apply to all its pathogen groups + "Failure frequency (days per year)": [t.failure_frequency] * 3, + "Failure duration (minutes)": [t.failure_duration] * 3, })] return pd.concat(dfs) diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv index 428fc08..f77a540 100644 --- a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv @@ -1,7 +1,7 @@ -Treatment,Pathogen group,Maximum LRV,Minimum LRV -Slow sand filtration,Viruses,4.0,0.25 -Slow sand filtration,Bacteria,6.0,2.0 -Slow sand filtration,Protozoa,5.0,0.3 -"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0 -"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6 -"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5 +Treatment,Pathogen group,Maximum LRV,Minimum LRV,Failure frequency (days per year),Failure duration (minutes) +Slow sand filtration,Viruses,4.0,0.25,0.0,30 +Slow sand filtration,Bacteria,6.0,2.0,0.0,30 +Slow sand filtration,Protozoa,5.0,0.3,0.0,30 +"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0,10.0,60 +"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6,10.0,60 +"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5,10.0,60 diff --git a/qmra/risk_assessment/tests/test_failure_inputs.py b/qmra/risk_assessment/tests/test_failure_inputs.py index 9a08bf6..f5517a0 100644 --- a/qmra/risk_assessment/tests/test_failure_inputs.py +++ b/qmra/risk_assessment/tests/test_failure_inputs.py @@ -4,6 +4,7 @@ from html.parser import HTMLParser from zipfile import ZipFile +import pandas as pd from assertpy import assert_that from django.test import TestCase from django.urls import reverse @@ -99,6 +100,9 @@ def new_configurator(client) -> dict: return data +TREATMENTS_CSV = "exposure-assessment/treatments.csv" + + def layout(csv: bytes) -> list[str]: """The lines of an exported table with every number replaced, so only its rows and columns remain.""" return [re.sub(r"-?\d+(\.\d+)?(e[-+]?\d+)?", "#", line) for line in csv.decode().splitlines()] @@ -233,13 +237,20 @@ def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): dict(name="Primary treatment", failure_frequency=0, failure_duration=30) ]) + @staticmethod + def exported_failure_inputs(export: dict) -> list[tuple]: + """(treatment, pathogen group, failure frequency, failure duration) of each row of the exported treatment table.""" + table = pd.read_csv(io.BytesIO(export[TREATMENTS_CSV])) + return list(table[["Treatment", "Pathogen group", "Failure frequency (days per year)", + "Failure duration (minutes)"]].itertuples(index=False, name=None)) + def saved_assessment_with_two_steps(self) -> RiskAssessment: data = new_configurator(self.client) add_step(data, "Primary treatment", **HIGH_LRVS) add_step(data, "Slow sand filtration", **HIGH_LRVS) return self.save_new(data) - def test_result_and_export_do_not_change_with_failure_frequency_0(self): + def test_failure_frequency_0_changes_neither_results_nor_export_beyond_the_durations(self): assessment = self.saved_assessment_with_two_steps() results_before = self.results(assessment) export_before = self.export(assessment) @@ -251,7 +262,24 @@ def test_result_and_export_do_not_change_with_failure_frequency_0(self): assert_that(failure_inputs_per_step(self.reopen(assessment))[0]["failure_duration"]).is_equal_to(120) assert_that(self.results(assessment)).is_equal_to(results_before) - assert_that(self.export(assessment)).is_equal_to(export_before) + # the export records the new durations in the treatment table; everything else stays identical + export_after = self.export(assessment) + durations = {duration for *_, duration in self.exported_failure_inputs(export_after)} + export_after.pop(TREATMENTS_CSV) + export_before.pop(TREATMENTS_CSV) + assert_that(export_after).is_equal_to(export_before) + assert_that(durations).is_equal_to({120, 1440}) + + def test_export_records_the_failure_inputs_of_each_step(self): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **HIGH_LRVS, failure_frequency="12.5", failure_duration="90") + add_step(data, "Slow sand filtration", **HIGH_LRVS) + assessment = self.save_new(data) + + groups = ["Viruses", "Bacteria", "Protozoa"] + assert_that(self.exported_failure_inputs(self.export(assessment))).is_equal_to( + [("Primary treatment", group, 12.5, 90) for group in groups] + + [("Slow sand filtration", group, 0, 30) for group in groups]) def test_export_keeps_its_files_and_columns_with_failures(self): assessment = self.saved_assessment_with_two_steps() From 305ec0f819f09af7896b35dce0870a4c2d6d5dda Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 25 Sep 2026 20:41:14 +0000 Subject: [PATCH 15/43] test: compare exported numbers to 10 significant digits (#30) The reference export test failed in CI: CI calculates with Python 3.11 and numpy 2.4.6, a developer machine with Python 3.12 and numpy 2.5.3, and the Monte Carlo results differ in their last digits. Decimal numbers in the compared files are now rounded to 10 significant digits before the comparison, and the reference is stored the same way. Every real change to a value, column or file is still caught. Checked with numpy 2.4.6 and 2.5.3. Refs #30 Co-Authored-By: Claude Opus 5.5 (1M context) --- .../risk_assessment/tests/reference_export/README.md | 3 +++ .../package/Reference assessment-report.html | 2 +- .../package/Reference assessment-result.csv | 12 ++++++------ .../package/exposure-assessment/inflows.csv | 2 +- .../package/exposure-assessment/treatments.csv | 12 ++++++------ qmra/risk_assessment/tests/test_reference_export.py | 9 +++++++-- 6 files changed, 24 insertions(+), 16 deletions(-) diff --git a/qmra/risk_assessment/tests/reference_export/README.md b/qmra/risk_assessment/tests/reference_export/README.md index d6a088d..de89670 100644 --- a/qmra/risk_assessment/tests/reference_export/README.md +++ b/qmra/risk_assessment/tests/reference_export/README.md @@ -8,6 +8,9 @@ and layout of the export package change only on purpose. - `package/` holds the exported tables and the HTML report. The report embeds the plots, which are masked as ``. The plots themselves are only checked for presence, because their rendering can differ between machines. +- Decimal numbers are stored and compared rounded to 10 significant digits. Numpy versions, + Python versions and processors differ in the last digits of the Monte Carlo results (CI + and a developer machine did); a real change to a value is far larger than that. The benchmark assessment's inputs are fixed values in the test, not bundled default data, so a new release of bundled source waters or treatment steps doesn't change the reference. The diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html index ab0890b..01b24e9 100644 --- a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html @@ -34,7 +34,7 @@

Result

  1. 1 in 10,000 infections per person per year
    2. -
  2. 0.000001 Disability Adjusted Life Years (DALY)
    3. +
  3. 1e-06 Disability Adjusted Life Years (DALY)
diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv index 36ceeb2..b0a0de5 100644 --- a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv @@ -4,9 +4,9 @@ "Result's Summary:" ,,Infection prob.,Infection prob.,Infection prob.,Infection prob.,Infection prob.,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy pathogen,stat,min,25%,50%,75%,max,min,25%,50%,75%,max -Rotavirus,Maximum LRV,9.572485276665432e-07,1.4836628104142324e-06,1.7097025438728153e-06,1.931930578930041e-06,2.893425146477391e-06,6.700739693665803e-09,1.0385639672899628e-08,1.1967917807109707e-08,1.3523514052510289e-08,2.0253976025341736e-08 -Rotavirus,Minimum LRV,0.3640096391082689,0.41122481268750266,0.4267809590038193,0.44364273411319005,0.5027473445430916,0.002548067473757882,0.002878573688812519,0.002987466713026735,0.0031054991387923303,0.003519231411801641 -Campylobacter jejuni,Maximum LRV,4.5133408121955654e-10,2.2749717942804892e-09,3.0147686569392818e-09,3.744822868467779e-09,7.008461921920173e-09,6.22841032082988e-13,3.139461076107075e-12,4.160380746576208e-12,5.167855558485534e-12,9.671677452249837e-12 -Campylobacter jejuni,Minimum LRV,0.0001083267586116099,0.0005457054173893527,0.0007230268318788591,0.0008980393643011064,0.0016799226351181806,1.4949092688402165e-07,7.530734759973068e-07,9.977770279928255e-07,1.2392943227355267e-06,2.3182932364630894e-06 -Cryptosporidium parvum,Maximum LRV,9.220223140538053e-09,1.429064969138416e-08,1.6467869567993176e-08,1.860837314793251e-08,2.7869510144107323e-08,9.681234297564956e-12,1.5005182175953368e-11,1.7291263046392836e-11,1.9538791805329135e-11,2.926298565131269e-11 -Cryptosporidium parvum,Minimum LRV,0.0028129655229737205,0.009066386874888782,0.011592701982251985,0.01415027039915931,0.025193602188789632,2.9536137991224066e-06,9.51970621863322e-06,1.2172337081364584e-05,1.4857783919117274e-05,2.645328229822911e-05 +Rotavirus,Maximum LRV,9.572485277e-07,1.48366281e-06,1.709702544e-06,1.931930579e-06,2.893425146e-06,6.700739694e-09,1.038563967e-08,1.196791781e-08,1.352351405e-08,2.025397603e-08 +Rotavirus,Minimum LRV,0.3640096391,0.4112248127,0.426780959,0.4436427341,0.5027473445,0.002548067474,0.002878573689,0.002987466713,0.003105499139,0.003519231412 +Campylobacter jejuni,Maximum LRV,4.513340812e-10,2.274971794e-09,3.014768657e-09,3.744822868e-09,7.008461922e-09,6.228410321e-13,3.139461076e-12,4.160380747e-12,5.167855558e-12,9.671677452e-12 +Campylobacter jejuni,Minimum LRV,0.0001083267586,0.0005457054174,0.0007230268319,0.0008980393643,0.001679922635,1.494909269e-07,7.53073476e-07,9.97777028e-07,1.239294323e-06,2.318293236e-06 +Cryptosporidium parvum,Maximum LRV,9.220223141e-09,1.429064969e-08,1.646786957e-08,1.860837315e-08,2.786951014e-08,9.681234298e-12,1.500518218e-11,1.729126305e-11,1.953879181e-11,2.926298565e-11 +Cryptosporidium parvum,Minimum LRV,0.002812965523,0.009066386875,0.01159270198,0.0141502704,0.02519360219,2.953613799e-06,9.519706219e-06,1.217233708e-05,1.485778392e-05,2.64532823e-05 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv index d2a8fbf..734d099 100644 --- a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv @@ -1,4 +1,4 @@ Pathogen,Minimum Concentration,Maximum Concentration Rotavirus,0.01,0.1 -Campylobacter jejuni,0.1,1.0 +Campylobacter jejuni,0.1,1 Cryptosporidium parvum,0.01,0.1 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv index f77a540..23568be 100644 --- a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv @@ -1,7 +1,7 @@ Treatment,Pathogen group,Maximum LRV,Minimum LRV,Failure frequency (days per year),Failure duration (minutes) -Slow sand filtration,Viruses,4.0,0.25,0.0,30 -Slow sand filtration,Bacteria,6.0,2.0,0.0,30 -Slow sand filtration,Protozoa,5.0,0.3,0.0,30 -"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0,10.0,60 -"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6,10.0,60 -"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5,10.0,60 +Slow sand filtration,Viruses,4,0.25,0,30 +Slow sand filtration,Bacteria,6,2,0,30 +Slow sand filtration,Protozoa,5,0.3,0,30 +"UV disinfection 20 mJ/cm2, drinking",Viruses,3,1,10,60 +"UV disinfection 20 mJ/cm2, drinking",Bacteria,6,4.6,10,60 +"UV disinfection 20 mJ/cm2, drinking",Protozoa,3,2.5,10,60 diff --git a/qmra/risk_assessment/tests/test_reference_export.py b/qmra/risk_assessment/tests/test_reference_export.py index 711aebe..384048e 100644 --- a/qmra/risk_assessment/tests/test_reference_export.py +++ b/qmra/risk_assessment/tests/test_reference_export.py @@ -26,13 +26,18 @@ def is_compared(name: str) -> bool: return not name.endswith("/") and name not in PLOTS +SIGNIFICANT_DIGITS = 10 + + def masked(name: str, content: bytes) -> str: - """The content of an exported file as it is compared; the plots embedded in the report are masked.""" + """The content of an exported file as it is compared: the plots embedded in the report are masked, and decimal + numbers are rounded to 10 significant digits. Numpy versions, Python versions and processors differ in the last + digits of the Monte Carlo results; every real change to a value is far larger than that.""" text = content.decode("utf-8") if name.endswith(".html"): text, plots = re.subn(r"base64,\s*[A-Za-z0-9+/=]+", "base64, ", text) assert_that(plots).described_as(f"plots masked in {name}").is_equal_to(len(PLOTS)) - return text + return re.sub(r"-?\d+\.\d+(?:e[-+]?\d+)?", lambda m: f"{float(m.group()):.{SIGNIFICANT_DIGITS}g}", text) class TestReferenceExport(TestCase): From 27b52687bb3767aa0982a6bc69237bc77f37239d Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 25 Sep 2026 20:46:26 +0000 Subject: [PATCH 16/43] test: compare exported numbers within 0.1 % instead of rounding (#30) Rounding to 10 significant digits was not enough: CI still differed in 30 of the 60 result values, all best-case, by up to 1.2e-5 relative. Tiny best-case probabilities (down to 1e-13 here) are computed as 1 - exp(-k * dose), where most digits cancel, so processors and math libraries differ far beyond the last digit. The difference is about 1e10 times smaller than the health-based reference level. The text of each compared file (names, columns, labels, layout) must stay identical, and each number must match within 0.1 % relative, about 100 times the largest deviation seen. CI's deviating value passes, and a value changed by 1 % or a renamed column still fails. The reference stores the full values again. Refs #30 Co-Authored-By: Claude Opus 5.5 (1M context) --- .../tests/reference_export/README.md | 7 +++--- .../package/Reference assessment-report.html | 2 +- .../package/Reference assessment-result.csv | 12 +++++----- .../package/exposure-assessment/inflows.csv | 2 +- .../exposure-assessment/treatments.csv | 12 +++++----- .../tests/test_reference_export.py | 22 +++++++++++++------ 6 files changed, 33 insertions(+), 24 deletions(-) diff --git a/qmra/risk_assessment/tests/reference_export/README.md b/qmra/risk_assessment/tests/reference_export/README.md index de89670..fc59aee 100644 --- a/qmra/risk_assessment/tests/reference_export/README.md +++ b/qmra/risk_assessment/tests/reference_export/README.md @@ -8,9 +8,10 @@ and layout of the export package change only on purpose. - `package/` holds the exported tables and the HTML report. The report embeds the plots, which are masked as ``. The plots themselves are only checked for presence, because their rendering can differ between machines. -- Decimal numbers are stored and compared rounded to 10 significant digits. Numpy versions, - Python versions and processors differ in the last digits of the Monte Carlo results (CI - and a developer machine did); a real change to a value is far larger than that. +- The text of each file must stay identical, and each number must match within 0.1 % + relative. Tiny best-case probabilities are computed as 1 − exp(−k · dose), where most + digits cancel, so machines with other processors or math libraries differ by up to about + 10⁻⁵ relative (seen between CI and a developer machine). A real change is far larger. The benchmark assessment's inputs are fixed values in the test, not bundled default data, so a new release of bundled source waters or treatment steps doesn't change the reference. The diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html index 01b24e9..ab0890b 100644 --- a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html @@ -34,7 +34,7 @@

Result

  1. 1 in 10,000 infections per person per year
    2. -
  2. 1e-06 Disability Adjusted Life Years (DALY)
    3. +
  3. 0.000001 Disability Adjusted Life Years (DALY)
diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv index b0a0de5..36ceeb2 100644 --- a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv @@ -4,9 +4,9 @@ "Result's Summary:" ,,Infection prob.,Infection prob.,Infection prob.,Infection prob.,Infection prob.,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy pathogen,stat,min,25%,50%,75%,max,min,25%,50%,75%,max -Rotavirus,Maximum LRV,9.572485277e-07,1.48366281e-06,1.709702544e-06,1.931930579e-06,2.893425146e-06,6.700739694e-09,1.038563967e-08,1.196791781e-08,1.352351405e-08,2.025397603e-08 -Rotavirus,Minimum LRV,0.3640096391,0.4112248127,0.426780959,0.4436427341,0.5027473445,0.002548067474,0.002878573689,0.002987466713,0.003105499139,0.003519231412 -Campylobacter jejuni,Maximum LRV,4.513340812e-10,2.274971794e-09,3.014768657e-09,3.744822868e-09,7.008461922e-09,6.228410321e-13,3.139461076e-12,4.160380747e-12,5.167855558e-12,9.671677452e-12 -Campylobacter jejuni,Minimum LRV,0.0001083267586,0.0005457054174,0.0007230268319,0.0008980393643,0.001679922635,1.494909269e-07,7.53073476e-07,9.97777028e-07,1.239294323e-06,2.318293236e-06 -Cryptosporidium parvum,Maximum LRV,9.220223141e-09,1.429064969e-08,1.646786957e-08,1.860837315e-08,2.786951014e-08,9.681234298e-12,1.500518218e-11,1.729126305e-11,1.953879181e-11,2.926298565e-11 -Cryptosporidium parvum,Minimum LRV,0.002812965523,0.009066386875,0.01159270198,0.0141502704,0.02519360219,2.953613799e-06,9.519706219e-06,1.217233708e-05,1.485778392e-05,2.64532823e-05 +Rotavirus,Maximum LRV,9.572485276665432e-07,1.4836628104142324e-06,1.7097025438728153e-06,1.931930578930041e-06,2.893425146477391e-06,6.700739693665803e-09,1.0385639672899628e-08,1.1967917807109707e-08,1.3523514052510289e-08,2.0253976025341736e-08 +Rotavirus,Minimum LRV,0.3640096391082689,0.41122481268750266,0.4267809590038193,0.44364273411319005,0.5027473445430916,0.002548067473757882,0.002878573688812519,0.002987466713026735,0.0031054991387923303,0.003519231411801641 +Campylobacter jejuni,Maximum LRV,4.5133408121955654e-10,2.2749717942804892e-09,3.0147686569392818e-09,3.744822868467779e-09,7.008461921920173e-09,6.22841032082988e-13,3.139461076107075e-12,4.160380746576208e-12,5.167855558485534e-12,9.671677452249837e-12 +Campylobacter jejuni,Minimum LRV,0.0001083267586116099,0.0005457054173893527,0.0007230268318788591,0.0008980393643011064,0.0016799226351181806,1.4949092688402165e-07,7.530734759973068e-07,9.977770279928255e-07,1.2392943227355267e-06,2.3182932364630894e-06 +Cryptosporidium parvum,Maximum LRV,9.220223140538053e-09,1.429064969138416e-08,1.6467869567993176e-08,1.860837314793251e-08,2.7869510144107323e-08,9.681234297564956e-12,1.5005182175953368e-11,1.7291263046392836e-11,1.9538791805329135e-11,2.926298565131269e-11 +Cryptosporidium parvum,Minimum LRV,0.0028129655229737205,0.009066386874888782,0.011592701982251985,0.01415027039915931,0.025193602188789632,2.9536137991224066e-06,9.51970621863322e-06,1.2172337081364584e-05,1.4857783919117274e-05,2.645328229822911e-05 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv index 734d099..d2a8fbf 100644 --- a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/inflows.csv @@ -1,4 +1,4 @@ Pathogen,Minimum Concentration,Maximum Concentration Rotavirus,0.01,0.1 -Campylobacter jejuni,0.1,1 +Campylobacter jejuni,0.1,1.0 Cryptosporidium parvum,0.01,0.1 diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv index 23568be..f77a540 100644 --- a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv @@ -1,7 +1,7 @@ Treatment,Pathogen group,Maximum LRV,Minimum LRV,Failure frequency (days per year),Failure duration (minutes) -Slow sand filtration,Viruses,4,0.25,0,30 -Slow sand filtration,Bacteria,6,2,0,30 -Slow sand filtration,Protozoa,5,0.3,0,30 -"UV disinfection 20 mJ/cm2, drinking",Viruses,3,1,10,60 -"UV disinfection 20 mJ/cm2, drinking",Bacteria,6,4.6,10,60 -"UV disinfection 20 mJ/cm2, drinking",Protozoa,3,2.5,10,60 +Slow sand filtration,Viruses,4.0,0.25,0.0,30 +Slow sand filtration,Bacteria,6.0,2.0,0.0,30 +Slow sand filtration,Protozoa,5.0,0.3,0.0,30 +"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0,10.0,60 +"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6,10.0,60 +"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5,10.0,60 diff --git a/qmra/risk_assessment/tests/test_reference_export.py b/qmra/risk_assessment/tests/test_reference_export.py index 384048e..44842e0 100644 --- a/qmra/risk_assessment/tests/test_reference_export.py +++ b/qmra/risk_assessment/tests/test_reference_export.py @@ -26,18 +26,27 @@ def is_compared(name: str) -> bool: return not name.endswith("/") and name not in PLOTS -SIGNIFICANT_DIGITS = 10 +NUMBER = re.compile(r"-?\d+(?:\.\d+)?(?:e[-+]?\d+)?") +# Tiny best-case probabilities are computed as 1 - exp(-k * dose), where most digits cancel, so machines with other +# processors or math libraries differ by up to about 1e-5 relative (seen between CI and a developer machine). +RELATIVE_TOLERANCE = 1e-3 def masked(name: str, content: bytes) -> str: - """The content of an exported file as it is compared: the plots embedded in the report are masked, and decimal - numbers are rounded to 10 significant digits. Numpy versions, Python versions and processors differ in the last - digits of the Monte Carlo results; every real change to a value is far larger than that.""" + """The content of an exported file as it is compared; the plots embedded in the report are masked.""" text = content.decode("utf-8") if name.endswith(".html"): text, plots = re.subn(r"base64,\s*[A-Za-z0-9+/=]+", "base64, ", text) assert_that(plots).described_as(f"plots masked in {name}").is_equal_to(len(PLOTS)) - return re.sub(r"-?\d+\.\d+(?:e[-+]?\d+)?", lambda m: f"{float(m.group()):.{SIGNIFICANT_DIGITS}g}", text) + return text + + +def assert_same_content(name: str, actual: str, expected: str): + """The text of a file identical, and each of its numbers within the relative tolerance.""" + assert_that(NUMBER.sub("#", actual)).described_as(name).is_equal_to(NUMBER.sub("#", expected)) + for value, reference in zip(NUMBER.findall(actual), NUMBER.findall(expected)): + assert_that(float(value)).described_as(f"{name}: {value} instead of {reference}").is_close_to( + float(reference), abs(float(reference)) * RELATIVE_TOLERANCE) class TestReferenceExport(TestCase): @@ -91,5 +100,4 @@ def test_export_package_matches_the_reference(self): assert_that(content).described_as(name).is_not_empty() elif is_compared(name): assert_that(str(PACKAGE / name)).described_as(f"reference of {name}").exists() - assert_that(masked(name, content)).described_as(name).is_equal_to( - (PACKAGE / name).read_text(encoding="utf-8")) + assert_same_content(name, masked(name, content), (PACKAGE / name).read_text(encoding="utf-8")) From faa21b2d44b05d66ccc57dae953ea85c2c94d535 Mon Sep 17 00:00:00 2001 From: yaniv-kwb Date: Mon, 28 Sep 2026 08:37:00 +0000 Subject: [PATCH 17/43] docs: C2 tickets 07-09 to in-progress; record the 2026-09-28 review in ticket 09 [skip ci] Co-Authored-By: Claude Fable 5.1 --- docs/tickets/07-c2-calculation-per-exposure-event.md | 2 +- docs/tickets/08-c2-failure-days-and-worst-case.md | 2 +- docs/tickets/09-c2-best-case-mixed-water.md | 10 +++++++++- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/tickets/07-c2-calculation-per-exposure-event.md b/docs/tickets/07-c2-calculation-per-exposure-event.md index 9f723d8..9026ec0 100644 --- a/docs/tickets/07-c2-calculation-per-exposure-event.md +++ b/docs/tickets/07-c2-calculation-per-exposure-event.md @@ -10,7 +10,7 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), Ch **Blocked by:** None (can start immediately) -**Status:** ready-for-agent +**Status:** in-progress (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) - [x] Every simulated exposure event has its own concentration sample, picked by position, and its own LRV. The random numbers are the same as today's. - [x] Best-case and worst-case share the same calculation steps. Summary statistics and the reference-level exceedance are computed in one place. diff --git a/docs/tickets/08-c2-failure-days-and-worst-case.md b/docs/tickets/08-c2-failure-days-and-worst-case.md index 2df600e..a1794a9 100644 --- a/docs/tickets/08-c2-failure-days-and-worst-case.md +++ b/docs/tickets/08-c2-failure-days-and-worst-case.md @@ -10,7 +10,7 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), AD **Blocked by:** 07 (#24), 04 (#21, C1 stored failure inputs) -**Status:** ready-for-agent +**Status:** in-progress (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) - [x] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. - [x] A step with failure frequency 365 gives the same worst-case results as the same scenario without that step's positive minimum LRVs. diff --git a/docs/tickets/09-c2-best-case-mixed-water.md b/docs/tickets/09-c2-best-case-mixed-water.md index 8e54c1e..2021c26 100644 --- a/docs/tickets/09-c2-best-case-mixed-water.md +++ b/docs/tickets/09-c2-best-case-mixed-water.md @@ -15,7 +15,7 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), AD **Blocked by:** 08 (#25) -**Status:** ready-for-agent +**Status:** in-progress (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) - [x] A step with failure frequency 365 and failure duration 1,440 gives the same best-case results as the same scenario without that step's positive maximum LRVs. - [x] A step with failure frequency 365 and failure duration 60 gives the same best-case results as the scenario with the train's best-case LRV replaced by the Eq. 5 mixed LRV, computed by hand in the test. @@ -30,3 +30,11 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), AD - Enter a scenario with failures and run it as a guest and as a registered user. - Confirm that the result page, the saved-assessments page, the assessment comparison and the export's result table show the changed results, with unchanged layout. - [ ] The calculation time for a drinking-water scenario (365 events per year) with failures is measured on the dev environment and recorded in the pull request. + +Review on 2026-09-28 (against the Spec, ADR-0001 and the code of the three C2 commits): + +- [x] The calculation matches the Spec: failure days per exposure event from a fixed-seed generator shared by both cases and all pathogens; worst-case loses only positive minimum LRVs; best-case follows the mixed-water rules including the three-or-more exception; LRVs of 0 or below are untouched. The hand-computed examples of decision 6 (mixed LRV about 6.9 and about 2.6) agree with the code's formula. +- [x] The concentration samples are picked by position with the same generator calls as before, so results without failures stay bit-identical. The PR records a 28-scenario check of this. +- [ ] Wording left over from before D4: `CONTEXT.md` (*Combined failure*: "loses all their LRVs"), `docs/vision.md` MVP item 3 and the C2 scope in `docs/failure_roadmap.md` still say worst-case loses the *full* LRV of a failing step. It should say every *positive* LRV. +- [ ] Decide whether it is acceptable that the failure draws of a step depend on its position in the train. Two scenarios that differ only by an extra non-failing step placed *before* a failing one give slightly different results, because the later step's random stream changes. The difference is sampling noise, but it can surprise a user comparing two saved assessments, and C5's independent script must use the same keying to match. +- [ ] The Spec still says "Status: draft" and "Spec issue: #___". Set it to implemented and create the Spec issue, or record that C2 has none. From 755baeb03a77276ce04fe58eca04cf83d3110209 Mon Sep 17 00:00:00 2001 From: yaniv-kwb Date: Mon, 28 Sep 2026 08:38:19 +0000 Subject: [PATCH 18/43] docs: worst-case loses every positive LRV of a failing step (D4 wording) [skip ci] Co-Authored-By: Claude Fable 5.1 --- CONTEXT.md | 2 +- docs/failure_roadmap.md | 2 +- docs/tickets/09-c2-best-case-mixed-water.md | 2 +- docs/vision.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 3263c81..5c82737 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -175,7 +175,7 @@ The best-case assumption that water consumed on a failure day is a mixture of wa _Avoid_: mixed concentration, dilution, storage effect **Combined failure**: -Failure events in two or more treatment steps on the same day. Worst-case loses all their LRVs for the whole day. Under the mixed-water assumption the failure events do not overlap if their durations fit into one day. Two failure events longer than a day in total overlap only by the minutes that exceed a day. Three or more failure events longer than a day in total count like worst-case: all these steps are down for the whole day. Only steps that lose LRV for the pathogen group count, so the same day can count like worst-case for one reference pathogen and not for another. +Failure events in two or more treatment steps on the same day. Worst-case loses all their positive LRVs for the whole day. Under the mixed-water assumption the failure events do not overlap if their durations fit into one day. Two failure events longer than a day in total overlap only by the minutes that exceed a day. Three or more failure events longer than a day in total count like worst-case: all these steps are down for the whole day. Only steps that lose LRV for the pathogen group count, so the same day can count like worst-case for one reference pathogen and not for another. **Next phase**: Capabilities identified for future development and not present in the application today. diff --git a/docs/failure_roadmap.md b/docs/failure_roadmap.md index 3dda3ef..7a9b002 100644 --- a/docs/failure_roadmap.md +++ b/docs/failure_roadmap.md @@ -31,7 +31,7 @@ Spec issue: #___ (draft: `docs/specs/c2-failure-calculation.md`) Purpose: results include treatment failures as decided in ADR-0001. Scope: failure days drawn per exposure event and per treatment step inside the Monte Carlo -simulation; worst-case loses the full LRV of every failing step; best-case applies the +simulation; worst-case loses every positive LRV of every failing step; best-case applies the mixed-water assumption; a failure removes only positive LRVs, LRVs of 0 or below stay (D4); combined failures that fit into a day do not overlap, two steps longer than a day overlap only by the extra minutes, and three or more steps longer than a day in total count diff --git a/docs/tickets/09-c2-best-case-mixed-water.md b/docs/tickets/09-c2-best-case-mixed-water.md index 2021c26..8287ab3 100644 --- a/docs/tickets/09-c2-best-case-mixed-water.md +++ b/docs/tickets/09-c2-best-case-mixed-water.md @@ -35,6 +35,6 @@ Review on 2026-09-28 (against the Spec, ADR-0001 and the code of the three C2 co - [x] The calculation matches the Spec: failure days per exposure event from a fixed-seed generator shared by both cases and all pathogens; worst-case loses only positive minimum LRVs; best-case follows the mixed-water rules including the three-or-more exception; LRVs of 0 or below are untouched. The hand-computed examples of decision 6 (mixed LRV about 6.9 and about 2.6) agree with the code's formula. - [x] The concentration samples are picked by position with the same generator calls as before, so results without failures stay bit-identical. The PR records a 28-scenario check of this. -- [ ] Wording left over from before D4: `CONTEXT.md` (*Combined failure*: "loses all their LRVs"), `docs/vision.md` MVP item 3 and the C2 scope in `docs/failure_roadmap.md` still say worst-case loses the *full* LRV of a failing step. It should say every *positive* LRV. +- [x] Wording left over from before D4 (fixed 2026-09-28): `CONTEXT.md` (*Combined failure*: "loses all their LRVs"), `docs/vision.md` MVP item 3 and the C2 scope in `docs/failure_roadmap.md` still say worst-case loses the *full* LRV of a failing step. It should say every *positive* LRV. - [ ] Decide whether it is acceptable that the failure draws of a step depend on its position in the train. Two scenarios that differ only by an extra non-failing step placed *before* a failing one give slightly different results, because the later step's random stream changes. The difference is sampling noise, but it can surprise a user comparing two saved assessments, and C5's independent script must use the same keying to match. - [ ] The Spec still says "Status: draft" and "Spec issue: #___". Set it to implemented and create the Spec issue, or record that C2 has none. diff --git a/docs/vision.md b/docs/vision.md index d63ce84..f3f3d5f 100644 --- a/docs/vision.md +++ b/docs/vision.md @@ -77,7 +77,7 @@ Functionalities, each one Change: 1. **Failure inputs per treatment step.** In the assessment configurator, each treatment step with at least one positive LRV gets a failure frequency (failure days per year, 0–365, default 0) and a failure duration (minutes, 1–1440, default 30). The inputs belong to the scenario. 2. **Failure inputs on personal treatment steps.** A personal treatment step can store its failure frequency and failure duration for reuse. -3. **Calculation with failure days.** Inside the Monte Carlo simulation, each exposure event falls on a failure day of each step with probability failure frequency / 365. Worst-case loses the full LRV of every failing step; best-case applies the mixed-water assumption: combined failures that fit into a day do not overlap, two steps longer than a day overlap only by the extra minutes, and three or more steps longer than a day in total count like worst-case. +3. **Calculation with failure days.** Inside the Monte Carlo simulation, each exposure event falls on a failure day of each step with probability failure frequency / 365. Worst-case loses every positive LRV of every failing step; best-case applies the mixed-water assumption: combined failures that fit into a day do not overlap, two steps longer than a day overlap only by the extra minutes, and three or more steps longer than a day in total count like worst-case. 4. **Results and export including failures.** The result page, the reference-level exceedance and the export package show the result including failures, and the export records the failure inputs of each step. Waits until later: From 5468a1f3ef4c9dd93bd30188c7000d149639319b Mon Sep 17 00:00:00 2001 From: yaniv-kwb Date: Mon, 28 Sep 2026 09:16:04 +0000 Subject: [PATCH 19/43] docs: C1 and C2 tickets 04-09 to review, matching the board [skip ci] Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/tickets/04-c1-failure-inputs-in-configurator.md | 2 +- ...1-failure-frequency-needs-positive-lrv-and-guest-messages.md | 2 +- docs/tickets/06-c1-personal-treatment-step-failure-inputs.md | 2 +- docs/tickets/07-c2-calculation-per-exposure-event.md | 2 +- docs/tickets/08-c2-failure-days-and-worst-case.md | 2 +- docs/tickets/09-c2-best-case-mixed-water.md | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/tickets/04-c1-failure-inputs-in-configurator.md b/docs/tickets/04-c1-failure-inputs-in-configurator.md index f8b3109..eaa2ee0 100644 --- a/docs/tickets/04-c1-failure-inputs-in-configurator.md +++ b/docs/tickets/04-c1-failure-inputs-in-configurator.md @@ -10,7 +10,7 @@ Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`), Change C1 in **Blocked by:** None (can start immediately) -**Status:** in-progress +**Status:** review - [x] The minimum ≤ maximum LRV check is defined once and used by both treatment step forms. Nothing changes that a user can notice, and the existing test suite passes unchanged. - [x] A schema migration adds failure frequency (a decimal number) and failure duration (a whole number) to the treatment steps of an assessment. Existing rows get failure frequency 0 and failure duration 30. diff --git a/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md b/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md index efb39d5..ddcc45f 100644 --- a/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md +++ b/docs/tickets/05-c1-failure-frequency-needs-positive-lrv-and-guest-messages.md @@ -10,7 +10,7 @@ Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`), roadmap deci **Blocked by:** 04 (#21) -**Status:** in-progress +**Status:** review - [x] A failure frequency above 0 is rejected, with a message on the failure frequency field, if none of the step's six LRVs (minimum and maximum for bacteria, viruses and protozoa) is above 0. An empty LRV counts as 0. - [x] A step whose only positive LRV is one maximum (for example protozoa 0–2) accepts a failure frequency above 0. diff --git a/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md b/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md index 5639aea..a369fb2 100644 --- a/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md +++ b/docs/tickets/06-c1-personal-treatment-step-failure-inputs.md @@ -10,7 +10,7 @@ Source: Spec C1 Failure inputs (`docs/specs/c1-failure-inputs.md`). The Spec iss **Blocked by:** 04 (#21), 05 (#22) -**Status:** in-progress +**Status:** review - [x] A schema migration adds failure frequency and failure duration to personal treatment steps. Existing personal treatment steps get failure frequency 0 and failure duration 30. - [x] The personal treatment step form shows both fields with their units and applies the same shared rule as the configurator: the ranges, whole minutes, D4 and minimum ≤ maximum LRV. Invalid input is rejected and the message is shown in the form. diff --git a/docs/tickets/07-c2-calculation-per-exposure-event.md b/docs/tickets/07-c2-calculation-per-exposure-event.md index 9026ec0..52c966f 100644 --- a/docs/tickets/07-c2-calculation-per-exposure-event.md +++ b/docs/tickets/07-c2-calculation-per-exposure-event.md @@ -10,7 +10,7 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), Ch **Blocked by:** None (can start immediately) -**Status:** in-progress (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) +**Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) - [x] Every simulated exposure event has its own concentration sample, picked by position, and its own LRV. The random numbers are the same as today's. - [x] Best-case and worst-case share the same calculation steps. Summary statistics and the reference-level exceedance are computed in one place. diff --git a/docs/tickets/08-c2-failure-days-and-worst-case.md b/docs/tickets/08-c2-failure-days-and-worst-case.md index a1794a9..070879b 100644 --- a/docs/tickets/08-c2-failure-days-and-worst-case.md +++ b/docs/tickets/08-c2-failure-days-and-worst-case.md @@ -10,7 +10,7 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), AD **Blocked by:** 07 (#24), 04 (#21, C1 stored failure inputs) -**Status:** in-progress (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) +**Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) - [x] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. - [x] A step with failure frequency 365 gives the same worst-case results as the same scenario without that step's positive minimum LRVs. diff --git a/docs/tickets/09-c2-best-case-mixed-water.md b/docs/tickets/09-c2-best-case-mixed-water.md index 8287ab3..5211614 100644 --- a/docs/tickets/09-c2-best-case-mixed-water.md +++ b/docs/tickets/09-c2-best-case-mixed-water.md @@ -15,7 +15,7 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), AD **Blocked by:** 08 (#25) -**Status:** in-progress (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) +**Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) - [x] A step with failure frequency 365 and failure duration 1,440 gives the same best-case results as the same scenario without that step's positive maximum LRVs. - [x] A step with failure frequency 365 and failure duration 60 gives the same best-case results as the scenario with the train's best-case LRV replaced by the Eq. 5 mixed LRV, computed by hand in the test. From 38c295e3fbcbfe72cc9d85f667b7c1a434335c5a Mon Sep 17 00:00:00 2001 From: yaniv-kwb Date: Mon, 28 Sep 2026 09:25:52 +0000 Subject: [PATCH 20/43] docs: C3 tickets 10-11 to review; record the 2026-09-28 review [skip ci] Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/tickets/10-c3-reference-export-test.md | 16 +++++++++----- .../tickets/11-c3-failure-inputs-in-export.md | 22 ++++++++++++------- 2 files changed, 25 insertions(+), 13 deletions(-) diff --git a/docs/tickets/10-c3-reference-export-test.md b/docs/tickets/10-c3-reference-export-test.md index 3942869..941626f 100644 --- a/docs/tickets/10-c3-reference-export-test.md +++ b/docs/tickets/10-c3-reference-export-test.md @@ -15,9 +15,15 @@ Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), the "Export sta **Blocked by:** None (can start immediately) -**Status:** ready-for-agent +**Status:** review (built in pull request #34, draft; CI green on `27b5268`; stays open until the release, D5) -- [ ] A test exports a fixed benchmark assessment and compares it with the stored reference export: the same files, identical CSV tables and HTML report, and both plots present. -- [ ] The test fails when a column, a file or a value of the export changes, and its message says which file differs. -- [ ] How to update the reference on purpose is documented next to the reference files. -- [ ] The export package itself is unchanged, and the existing test suite passes. +- [x] A test exports a fixed benchmark assessment and compares it with the stored reference export: the same files, identical CSV tables and HTML report, and both plots present. +- [x] The test fails when a column, a file or a value of the export changes, and its message says which file differs. +- [x] How to update the reference on purpose is documented next to the reference files. +- [x] The export package itself is unchanged, and the existing test suite passes. + +Review on 2026-09-28 (commits `7085d78`, `305ec0f`, `27b5268`): + +- [x] The test exports a fixed benchmark assessment with fixed inputs, not bundled data, and one failing step, so it also covers the C2 calculation. Plots are checked for presence only and masked inside the report. +- [x] The comparison is not byte-identical, as this ticket first said: text must be identical and each number within 0.1 % relative. The reason is recorded in the README and the PR: tiny probabilities from 1 − exp(−k · dose) differ by up to 10⁻⁵ between CI and the server. A renamed column, a missing file or a 1 % change still fails. +- [ ] Follow-up, not part of C3: computing those probabilities with `expm1` would remove the cancellation and allow a tighter tolerance. That is a change to the calculation and needs its own ticket if wanted. diff --git a/docs/tickets/11-c3-failure-inputs-in-export.md b/docs/tickets/11-c3-failure-inputs-in-export.md index dee86b1..cca3996 100644 --- a/docs/tickets/11-c3-failure-inputs-in-export.md +++ b/docs/tickets/11-c3-failure-inputs-in-export.md @@ -10,11 +10,17 @@ Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), decisions of 20 **Blocked by:** 10 (#30) -**Status:** ready-for-agent - -- [ ] `treatments.csv` has the columns "Failure frequency (days per year)" and "Failure duration (minutes)", with each step's values on all three of its rows. -- [ ] An assessment saved with failure inputs exports exactly those values per step, and an assessment without failure inputs exports 0 and 30. -- [ ] No other file of the export package changes; the HTML report shows no inputs, as before. -- [ ] The reference export is updated in the same commit, and the reference test passes. -- [ ] The roadmap's C3 entry says that the HTML report is left out because it shows no inputs. -- [ ] The pull request notes the export change for the release notes. +**Status:** review (built in pull request #34, draft; CI green on `27b5268`; stays open until the release, D5) + +- [x] `treatments.csv` has the columns "Failure frequency (days per year)" and "Failure duration (minutes)", with each step's values on all three of its rows. +- [x] An assessment saved with failure inputs exports exactly those values per step, and an assessment without failure inputs exports 0 and 30. +- [x] No other file of the export package changes; the HTML report shows no inputs, as before. +- [x] The reference export is updated in the same commit, and the reference test passes. +- [x] The roadmap's C3 entry says that the HTML report is left out because it shows no inputs. +- [x] The pull request notes the export change for the release notes. + +Review on 2026-09-28 (commit `a04e2f9`): + +- [x] Only `treatments.csv` changed in the reference export, in the same commit as the code; later commits touched only the test tolerance and the README. +- [x] Assessments saved before C1 export 0 and 30, because the new fields have those defaults. +- [ ] Manual check on dev from PR #34 still open: download an export with failure inputs, check the two columns and the unchanged report. From 0a550ef203367555ddd089fc9de44a50dc7cf697 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Tue, 29 Sep 2026 07:52:37 +0000 Subject: [PATCH 21/43] docs: record the progress of the failure work C1-C5 [skip ci] Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/progress.md | 101 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 docs/progress.md diff --git a/docs/progress.md b/docs/progress.md new file mode 100644 index 0000000..a5bb425 --- /dev/null +++ b/docs/progress.md @@ -0,0 +1,101 @@ +# Progress: treatment failure (C1–C5) + +Status: 2026-09-29. Work of 2026-09-24 to 2026-09-28 on the Changes in `docs/failure_roadmap.md`. +Nothing is merged into `main` (roadmap decision D5): each Change is a draft pull request that +targets `main` only so that the pipeline deploys it to dev. The production jobs run only for +`main` itself. + +## Overview + +| Change | Branch (based on) | Pull request | Tickets | State | +|---|---|---|---|---| +| C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`) | Built, tested manually on dev | +| C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | Built, tested manually on dev | +| C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open | +| C4 Explain treatment failure | not created yet (will be based on C2) | — | #32, #33 (`12`, `13`) | Tickets ready | +| C5 Failure benchmark | — | — | — | Waits for decisions D1–D3 | + +**Dev** (https://dev.qmra.org) runs C1 + C2 + C3 (PR #34, commit `27b5268`) since 2026-09-25. + +**Review of 2026-09-28** (by a teammate): +- **Statuses:** tickets 04–11 are set to "review". +- **Review notes:** ticket 09 records a review of the C2 commits against the Spec, ADR-0001 and the code; tickets 10 and 11 record a review of the C3 commits. +- **D4 wording:** `CONTEXT.md`, the roadmap and the vision now say that worst-case loses every *positive* LRV of a failing step, as the code does. + +## What each Change delivers + +### C1 Failure inputs (#28) + +- **Configurator (#21, `57bdd21`):** + - each treatment step has a failure frequency (days per year, 0–365, default 0) and a failure duration (whole minutes, 1–1440, default 30), with units; + - values are checked on run and save, and stay with their step through save, reopen and edit; + - the migration gives existing assessments 0 and 30; + - one shared validation rule replaces the duplicated min ≤ max LRV check. +- **Positive LRV and guests (#22, `52e6087`):** + - a frequency above 0 needs at least one positive LRV (D4); + - guests now see validation messages on the affected field and step (before, an invalid guest result failed silently); + - the treatments section grows with its content instead of hiding the new fields under the footer. +- **Personal treatment steps (#23, `7f05bcd`):** they store both values under the same rules, the configurator copies them into a newly added step, and the copy can be changed without touching the personal step. + +### C2 Failure calculation (#29) + +- **Tidy-up (#24, `93c8f76`):** the simulation works per exposure event. Results are bit-identical to before (checked on 28 scenarios, 1932 values). +- **Failure days and worst-case (#25, `2f6efec`):** + - each exposure event falls on a failure day of each step with probability frequency / 365 (ADR-0001); + - the draws come from their own fixed seed, one stream per step position, shared by both cases and all pathogens; + - worst-case loses the failing steps' positive minimum LRVs for the whole day. +- **Best-case (#26, `a8a1972`):** the mixed-water assumption. For combined failures of one day, only steps that lose LRV for the pathogen group count: + - failures that fit into a day don't overlap (one step: Eq. 5); + - two steps longer than a day overlap by the extra minutes; + - three or more steps longer than a day in total count like worst-case. +- **Without failures:** results are exactly those from before C2. +- **Local timing** (drinking water, 3 pathogens, 5 bundled steps): 0.07 s without failures, 0.13–0.19 s with failures. + +### C3 Failure inputs in the export (#34) + +- **Reference export test (#30, `7085d78`, fixes `305ec0f`, `27b5268`):** implements the "Export stability" Quality goal. + - A fixed benchmark assessment is exported and compared with `qmra/risk_assessment/tests/reference_export/`: the same files, identical text, and each number within 0.1 % relative. Plots are only checked for presence. + - `UPDATE_REFERENCE_EXPORT=1` rewrites the reference on purpose (see the README there). +- **Failure columns (#31, `a04e2f9`):** `treatments.csv` has "Failure frequency (days per year)" and "Failure duration (minutes)", repeated on each step's three rows. The HTML report stays without inputs. +- **For the release notes:** `treatments.csv` has two new columns at the end. + +## Decisions made during implementation + +All on 2026-09-25 unless noted. The documents named are updated. + +- **C1: failure fields are required in the form.** Existing form tests got the two fields in their input data. +- **C1: guests see validation errors.** The guest result returns the messages, and the configurator shows them on the affected step (option a). +- **C1: dev only moves forward.** The migrations can't be reversed on dev, and only C1 and branches built on it go there until the release. The columns get no database default. +- **C2: statistics instead of the mean.** Tests check the returned statistics and the exceedance, because the mean isn't returned. Per-year coupling makes that equivalent. +- **C2: one C1 test split.** With frequency 0, results and export stay identical. With failures, only values change. +- **C2: three or more steps over a day count like worst-case in best-case.** This replaces the lowest-concentration arrangement decided on 2026-09-24, which needed a linear program whose solver failed for large combined losses; the per-pathogen arrangement rule is dropped with it. See the C2 Spec, `CONTEXT.md` (*Combined failure*), decision 6 in `docs/QMRA_Failure_Approach.md`, the roadmap and the vision. +- **C2: only steps that lose LRV for the pathogen group count** towards "three or more" (Reading 1). +- **C2: tested on dev with a PR into `main`,** because the pipeline deploys only pull requests into `main`. +- **C3 and C4 have no Spec** (Small Changes). The Tickets follow the roadmap entries. +- **C3: the HTML report shows no inputs** and doesn't get failure inputs. Roadmap entry corrected. +- **C3: failure columns are repeated on each step's three rows** of `treatments.csv`. +- **C3: a reference export test is added**, comparing numbers within 0.1 %. CI's processor and math library differ from this server's by up to 10⁻⁵ relative in tiny best-case probabilities. +- **C4: texts are drafted by Claude** and reviewed by a domain expert before the release. Step 4 of the guided tour is extended, and a new Sphinx page is added. +- **C3 and C4 are based on C2,** so dev keeps C1 + C2 while they are tested. + +## Open items + +**Checks still to do** +- **C1 (#28):** real-data check; "I can explain this Change". +- **C2 (#29):** calculation time measured on dev and recorded in the PR (ticket 09, #26); real-data check; "I can explain this Change". +- **C3 (#34):** manual check on dev, from the list in the PR; real-data check; "I can explain this Change". + +**Work not started** +- **C4:** #32 (FAQ and guided tour) and #33 (Sphinx page) can start now. +- **C5:** needs decisions D1 (tolerance), D2 (who writes the independent script) and D3 (who signs off). The script must implement the combined-failure rule of 2026-09-25. + +**Known issues and risks** +- **Bug #27** (older than C1): an empty "Events per year" or "Volume per event" gives a server error for guests and registered users. +- **Lost click** (existing behaviour): typing into a field and clicking "Show assessment results" straight away loses the first click. +- **Precision:** tiny best-case probabilities lose precision in `1 − exp(−k·dose)`. Making the calculation more precise (`expm1`/`log1p`) would be a separate change to the calculation. +- **Step order:** failure draws are keyed by a step's position in the train. Saved assessments pass their steps without an explicit order (stable in practice). +- **Unpinned dependencies:** numpy and pandas aren't pinned; reproducibility across releases is an open question for Architecture. +- **Housekeeping:** + - the C1 and C2 Spec issues were never created (`#___` in the roadmap); + - the local branch `c2-failure-calculation` on the Controlled server is behind GitHub (the review commits of 2026-09-28) and holds one ticket commit (`5ca99b1`) that is on GitHub only through `c3-failure-inputs-in-export`. Update it from GitHub before working on it again. + - The GitHub command-line login on the Controlled server has expired (`HTTP 401`); issues and pull requests can't be updated from there until someone logs in again. From e9d5ce71e27554d4222e051cf83a8dc212d29a9a Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 16 Oct 2026 15:30:17 +0000 Subject: [PATCH 22/43] docs: revise the C1 spec for D7, number of parallel units [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/specs/c1-failure-inputs.md | 328 +++++++++++++++++++------------- 1 file changed, 201 insertions(+), 127 deletions(-) diff --git a/docs/specs/c1-failure-inputs.md b/docs/specs/c1-failure-inputs.md index c8e50af..f609f0e 100644 --- a/docs/specs/c1-failure-inputs.md +++ b/docs/specs/c1-failure-inputs.md @@ -1,32 +1,47 @@ # Spec C1: Failure inputs -Status: draft, 2026-09-24. Change C1 in `docs/failure_roadmap.md` (Large). Spec issue: #___ -Sources: `docs/vision.md` (MVP item 1 and 2), `CONTEXT.md`, ADR-0001, roadmap decisions D4 -and D5. +Status: draft, 2026-09-24, revised 2026-10-16 for D7 (number of parallel units). Change C1 +in `docs/failure_roadmap.md` (Large). Spec issue: #___ +Sources: `docs/vision.md` (MVP item 1 and 2), `CONTEXT.md`, ADR-0001, ADR-0002, roadmap +decisions D4, D5 and D7. + +The first version (failure frequency and failure duration) is built and was tested on the +dev environment (tickets 04–06). This revision adds the third failure input, the number of +parallel units, and keeps everything else. Requirements that are new in this revision are +marked *(D7)*. ## Problem Statement A utility operator or engineer who wants to know how failures of their treatment steps -affect the yearly risk has nowhere to say how often a step fails and for how long. The -assessment configurator only knows LRVs, and a personal treatment step only stores a name -and LRVs. Before the calculation can include treatment failures (C2), users need a place to -enter a failure frequency and a failure duration for each treatment step, and the -application needs to keep those values with the scenario. +affect the yearly risk has nowhere to say how often a step fails, for how long, and how much +of the step stops when it fails. The assessment configurator only knew LRVs, and a personal +treatment step only stored a name and LRVs. Before the calculation can include treatment +failures (C2), users need a place to enter a failure frequency, a failure duration and a +number of parallel units for each treatment step, and the application needs to keep those +values with the scenario. + +In most plants the LRV of a step does not drop to 0 during a failure: a step often has +several parallel units, and a failure event stops only one of them (D7, ADR-0002). Without +the number of parallel units, every failure would be counted as a complete loss of the +step's positive LRVs. ## Solution -Every treatment step in the assessment configurator gets two more fields next to its LRVs: -**failure frequency** (failure days per year) and **failure duration** (minutes). They -start at 0 days per year and 30 minutes, so a step does not fail unless the user says so. -The application checks the values when the assessment is run or saved and explains what is -wrong. A step can only have a failure frequency above 0 if at least one of its LRVs is -positive, since a failure removes only positive LRVs (D4). +Every treatment step in the assessment configurator has three failure fields next to its +LRVs: **failure frequency** (failure days per year), **failure duration** (minutes) and +**number of parallel units** *(D7)*. They start at 0 days per year, 30 minutes and 1 unit, +so a step does not fail unless the user says so, and a step with one unit loses all its +positive LRVs when it fails, as before. The application checks the values when the +assessment is run or saved and explains what is wrong. A step can only have a failure +frequency above 0 if at least one of its LRVs is positive, since a failure removes only +positive LRVs (D4). -Registered users can also store both values on a personal treatment step. When they add -that step to an assessment, its failure frequency and failure duration are copied in like -its LRVs. Saved assessments keep the values, and reopening or editing an assessment shows -them again. Assessments saved before this change reopen with failure frequency 0 and -failure duration 30, so nothing about them changes. +Registered users can also store the three values on a personal treatment step. When they +add that step to an assessment, its failure inputs are copied in like its LRVs. Saved +assessments keep the values, and reopening or editing an assessment shows them again. +Assessments and personal treatment steps saved before a failure input existed read back its +default: failure frequency 0, failure duration 30 and 1 parallel unit, so nothing about +them changes. In this Change the values do not yet affect the result. The calculation follows in C2, and nothing reaches production before C1–C5 are released together (D5). @@ -37,109 +52,142 @@ nothing reaches production before C1–C5 are released together (D5). assessment, so that the assessment reflects how often each step fails at my plant. 2. As a utility operator, I want to enter a failure duration for each treatment step, so that the assessment reflects how long a typical failure event lasts. -3. As a user, I want the failure frequency to be given in failure days per year, so that I +3. As a utility operator, I want to enter the number of parallel units of each treatment + step, so that a failure of one unit is not counted as a failure of the whole step. + *(D7)* +4. As a user, I want the failure frequency to be given in failure days per year, so that I can use the numbers from my plant's operating records directly. -4. As a user, I want the failure duration to be given in minutes, so that short failure +5. As a user, I want the failure duration to be given in minutes, so that short failure events can be expressed without fractions. -5. As a user, I want the unit of each failure field shown next to it, so that I do not - enter hours instead of minutes or events instead of days. -6. As a user, I want every treatment step to start with failure frequency 0, so that an +6. As a user, I want the number of parallel units to be a plain count of units, so that I + can take it from the plant layout. *(D7)* +7. As a user, I want the unit or meaning of each failure field shown in its label, so that I + do not enter hours instead of minutes, events instead of days, or a percentage instead of + a count of units. +8. As a user, I want every treatment step to start with failure frequency 0, so that an assessment without failure information behaves as before. -7. As a user, I want every treatment step to start with failure duration 30 minutes, so +9. As a user, I want every treatment step to start with failure duration 30 minutes, so that I only change the duration when I know a better value. -8. As a user, I want to enter a fractional failure frequency such as 0.5 days per year, so - that I can describe a step that fails once every two years. -9. As a user, I want a failure frequency below 0 or above 365 to be rejected with a clear - message, so that I notice a typing mistake. -10. As a user, I want a failure duration below 1 or above 1440 minutes, or a non-whole +10. As a user, I want every treatment step to start with 1 parallel unit, so that a step I do + not describe further loses all its positive LRVs during a failure, as before. *(D7)* +11. As a user, I want to enter a fractional failure frequency such as 0.5 days per year, so + that I can describe a step that fails once every two years. +12. As a user, I want a failure frequency below 0 or above 365 to be rejected with a clear + message, so that I notice a typing mistake. +13. As a user, I want a failure duration below 1 or above 1440 minutes, or a non-whole number of minutes, to be rejected with a clear message, so that I notice a typing mistake or a wrong unit. -11. As a user, I want to be told when I give a failure frequency above 0 to a step that has +14. As a user, I want a number of parallel units below 1, or one that is not a whole number, + to be rejected with a clear message, so that I notice a typing mistake. *(D7)* +15. As a user of a large plant, I want to enter any number of parallel units, however large, + so that the application does not stop me from describing my plant. *(D7)* +16. As a user, I want to be told when I give a failure frequency above 0 to a step that has no positive LRV, so that I understand that such a step cannot fail. -12. As a user modelling recontamination with a negative LRV, I want that step to keep +17. As a user modelling recontamination with a negative LRV, I want that step to keep failure frequency 0, so that recontamination is never treated as a failing step. -13. As a user, I want a step with a positive LRV for one pathogen group and a zero or +18. As a user, I want a step with a positive LRV for one pathogen group and a zero or negative LRV for another to accept failure inputs, so that steps like chlorination (protozoa minimum 0) or a storage tank with regrowth can fail. -14. As a user, I want the failure fields to appear in the same card as the step's LRVs, so +19. As a user, I want the number of parallel units shown for every treatment step, next to + failure frequency and failure duration, so that I see all failure inputs of a step + together and the form does not change shape while I type. *(D7)* +20. As a user describing a non-technical measure such as a hygiene practice, I want the same + three failure fields, so that I can leave the number of units at 1 and the measure is + lost completely during a failure. *(D7)* +21. As a user whose units fail together, for example through a shared power supply, I want + to enter 1 unit, so that such a failure is counted as a loss of the whole step. *(D7)* +22. As a user, I want the failure fields to appear in the same card as the step's LRVs, so that I see all values of a step together. -15. As a user, I want the failure values of each step to stay with that step when I - reorder or remove other steps, so that values do not move to the wrong step. -16. As a user adding the same bundled step twice, I want to give each copy its own failure - values, so that two parallel units can have different failure records. -17. As a guest, I want to enter failure inputs and run an assessment, so that I can try the - feature without an account. -18. As a guest, I want invalid failure inputs to be rejected in the same way as for +23. As a user, I want the failure values of each step to stay with that step when I reorder + or remove other steps, so that values do not move to the wrong step. +24. As a user adding the same bundled step twice, I want to give each copy its own failure + values, so that I can describe two separate installations of the same technology. +25. As a guest, I want to enter all three failure inputs and run an assessment, so that I + can try the feature without an account. +26. As a guest, I want invalid failure inputs to be rejected in the same way as for registered users, so that the rules do not depend on having an account. -19. As a registered user, I want the failure values of each step saved with my assessment, +27. As a registered user, I want the failure values of each step saved with my assessment, so that I do not have to enter them again. -20. As a registered user, I want to see the saved failure values when I reopen an +28. As a registered user, I want to see the saved failure values when I reopen an assessment, so that I can check what the assessment is based on. -21. As a registered user, I want to change failure values when I edit an assessment and +29. As a registered user, I want to change failure values when I edit an assessment and have the new values saved, so that I can correct or update them. -22. As a registered user, I want assessments I saved before this change to reopen with - failure frequency 0 and failure duration 30, so that their results do not change. -23. As a registered user, I want to store a failure frequency and failure duration on a - personal treatment step, so that my plant's failure record is reused in every - assessment that uses the step. -24. As a registered user, I want the same checks on a personal treatment step as in the +30. As a registered user, I want assessments I saved before a failure input existed to + reopen with its default (failure frequency 0, failure duration 30, 1 parallel unit), so + that their results do not change. +31. As a registered user, I want to store all three failure inputs on a personal treatment + step, so that my plant's failure record is reused in every assessment that uses the + step. +32. As a registered user, I want the same checks on a personal treatment step as in the configurator, so that a personal step never carries values an assessment would reject. -25. As a registered user, I want a personal treatment step's failure values copied into the - assessment when I add the step, so that I do not type them again. -26. As a registered user, I want to change the copied failure values in one assessment +33. As a registered user, I want a personal treatment step's failure values, including its + number of parallel units, copied into the assessment when I add the step, so that I do + not type them again. +34. As a registered user, I want to change the copied failure values in one assessment without changing the personal treatment step, so that I can try other values in one scenario. -27. As a registered user, I want personal treatment steps I created before this change to - offer failure frequency 0 and failure duration 30, so that they keep working. -28. As a registered user, I want my personal treatment steps and their failure values to be +35. As a registered user, I want personal treatment steps I created before a failure input + existed to offer its default, so that they keep working. +36. As a registered user, I want my personal treatment steps and their failure values to be visible only to me, so that my plant's failure record stays private. -29. As a user comparing scenarios, I want two saved assessments to differ only in their - failure inputs, so that I can later compare the effect of different failure - frequencies. -30. As a user, I want bundled treatment steps to come without failure values, so that I am +37. As a user comparing scenarios, I want two saved assessments to differ only in their + failure inputs, for example only in the number of parallel units of one step, so that I + can later compare the effect of that input. +38. As a user, I want bundled treatment steps to come without failure values, so that I am not given literature failure data the application does not have. -31. As a developer of C2, I want the failure frequency and failure duration of every step in - an assessment to be available where the calculation reads the treatment train, so that - C2 only has to change the calculation. -32. As a developer of C3, I want the stored failure values to be readable from a saved - assessment, so that C3 can add them to the export package and report. +39. As a developer of C2, I want the failure frequency, failure duration and number of + parallel units of every step in an assessment to be available where the calculation + reads the treatment train, so that C2 only has to change the calculation. +40. As a developer of C3, I want the stored failure values to be readable from a saved + assessment, so that C3 can add them to the export package. ## Implementation Decisions -- **Two new attributes on two kinds of treatment step.** The treatment step of an - assessment and the personal treatment step each get *failure frequency* (a decimal - number of failure days per year) and *failure duration* (a whole number of minutes). - Both are required, with defaults 0 and 30. A schema migration adds them to both, and - existing rows receive the defaults. +- **Three attributes on two kinds of treatment step.** The treatment step of an assessment + and the personal treatment step each carry *failure frequency* (a decimal number of + failure days per year), *failure duration* (a whole number of minutes) and *number of + parallel units* (a whole number) *(D7)*. All three are required, with defaults 0, 30 and + 1. Failure frequency and failure duration exist since tickets 04–06. A new schema + migration adds the number of parallel units to both kinds of step, and existing rows + receive 1. - **Bundled treatment steps are unchanged.** They carry no failure data (failure inputs are user input only). A bundled step added to an assessment starts at the defaults. - **One validation rule, used by both forms.** The treatment step form of the configurator - and the personal treatment step form apply the same rule, defined once: + and the personal treatment step form apply the same shared rule, which already checks + minimum ≤ maximum LRV and the first two failure inputs. It is extended, not duplicated: - failure frequency from 0 to 365 inclusive, decimals allowed; - failure duration a whole number from 1 to 1440 inclusive; + - number of parallel units a whole number of 1 or more, with no upper limit *(D7)*; - failure frequency above 0 only if at least one of the step's six LRVs (minimum and maximum for bacteria, viruses and protozoa) is above 0; an empty LRV counts as 0 (D4). The error is attached to the failure frequency field. - The duration range is checked even when the frequency is 0, so stored values are always - valid. The existing minimum ≤ maximum LRV check, which both forms currently duplicate, - may move into the same shared rule. -- **Configurator.** Each treatment step card shows the two fields below its LRV rows, - labelled with their units ("Failure frequency (days per year)", "Failure duration - (minutes)"). The fields are always shown; the D4 rule is enforced by validation, since the - user can change LRVs after entering a frequency. Guests see and use the same fields. -- **Saving and reopening.** The treatment step formset saves the two values with each step, - and reopening an assessment fills them in again. Each value belongs to its step, not to a - position in the train. -- **Personal treatment steps.** The personal treatment step form gets the two fields. The - list of personal treatment steps that the configurator loads includes both values, and - the configurator copies them into a newly added step together with its LRVs. The copy is - independent: editing it in an assessment leaves the personal treatment step unchanged. + The ranges are checked even when the frequency is 0, so stored values are always valid. + The messages follow the existing pattern and name the field and its range, for example + "number of parallel units must be a whole number of 1 or more". +- **Configurator.** Each treatment step card shows the three failure fields below its LRV + rows, labelled "Failure frequency (days per year)", "Failure duration (minutes)" and + "Number of parallel units". The fields are always shown, also when the failure frequency + is 0; the D4 rule is enforced by validation, since the user can change LRVs after + entering a frequency. Guests see and use the same fields. The labels say nothing more; + what a parallel unit is, that the failure duration affects both best-case and + worst-case, and that units failing together are entered as 1 unit are explained in C4. +- **Saving and reopening.** The treatment step formset saves the three values with each + step, and reopening an assessment fills them in again. Each value belongs to its step, + not to a position in the train. +- **Personal treatment steps.** The personal treatment step form gets the number of + parallel units next to the two existing failure fields. The list of personal treatment + steps that the configurator loads includes all three values, and the configurator copies + them into a newly added step together with its LRVs. The copy is independent: editing it + in an assessment leaves the personal treatment step unchanged. - **Guest path.** The guest result submits the same treatment step formset. Failure values are validated there too and are not stored. - **No effect on results.** The calculation, the result page and the export package are not - changed in C1. The values are stored so that C2 and C3 can use them. -- **Delivery.** C1 stays an open pull request and is tested on the dev environment; it is - not merged into `main` before the release (D5). C2 and C3 build on it. + changed in C1. The values are stored so that C2 and C3 can use them; C2 reads the number + of parallel units to calculate the failure LRV (ADR-0002). +- **Delivery.** The number of parallel units is added in new tickets on the existing C1 pull + request (roadmap decision "Rework of C1–C3"). C1 stays an open pull request and is tested + on the dev environment again; it is not merged into `main` before the release (D5). C2 + and C3 build on it. ## Testing Decisions @@ -150,71 +198,94 @@ nothing reaches production before C1–C5 are released together (D5). - **One seam: HTTP through the Django test client.** Tests post the configurator form (save and guest result) and the personal treatment step form, then read the saved assessment, the reopened configurator or the personal treatment step list. This covers form - validation, saving, reopening and ownership without a separate form-level seam. -- **Behaviour covered at that seam:** - - saving an assessment with failure inputs and reopening it returns the same values; - editing and saving again stores the new values; + validation, saving, reopening and ownership without a separate form-level seam. The + number of parallel units is tested at the same seam as the other two failure inputs. +- **Behaviour covered at that seam** (cases new in this revision marked *(D7)*): + - a new step card, the guest form and the personal treatment step form show all three + failure fields with their labels and defaults 0, 30 and 1 *(D7: the third field)*; + - saving an assessment with failure inputs and reopening it returns the same values, + including the number of parallel units; editing and saving again stores the new values, + per step *(D7)*; - out-of-range frequency or duration, a non-whole duration, and a frequency above 0 on a step without a positive LRV are rejected with a message, on the saved configurator, on the guest result and on the personal treatment step form; + - a number of parallel units of 0, a negative number or a fraction such as 1.5 is rejected + with a message on all three forms; 1, 2 and a large value such as 1000 are accepted + *(D7)*; - a step whose only positive LRV is one maximum (for example protozoa 0–2) accepts a frequency above 0; a step with only negative or empty LRVs does not; - a guest result with valid failure inputs is accepted; - - a personal treatment step saved with failure inputs appears with them in its owner's - list; another user's list does not contain it; - - an assessment and a personal treatment step created without failure values (as before - the migration) come back with frequency 0 and duration 30; - - the result of an assessment is identical with and without failure inputs (C1 does not - change the calculation). -- **Prior art.** The form tests in the risk-assessment test module (min ≤ max, negative LRVs - allowed) show the validation cases to extend; the calculation regression test shows how - fixed results are compared. There are no HTTP-level tests yet; C1 adds the first ones, - using Django's test client and a logged-in test user. + - a personal treatment step saved with failure inputs appears with all three in its + owner's list; another user's list does not contain it *(D7: the third value)*; + - an assessment and a personal treatment step created without the number of parallel + units (as before its migration) come back with 1 *(D7)*, and without any failure values + with frequency 0, duration 30 and 1 unit; + - the result and the export package of an assessment are identical with and without + failure inputs, including a number of parallel units above 1 (C1 does not change the + calculation) *(D7)*. +- **Prior art.** The C1 HTTP tests in the risk-assessment test package (failure inputs on + the configurator, guest result and personal treatment steps, with a logged-in test user + and a parser for the configurator form) are extended with the cases above. The form tests + in the same package show the minimum ≤ maximum LRV cases; the calculation regression test + shows how fixed results are compared. - **Manual check on the dev environment.** Copying a personal treatment step's failure values into the configurator happens in the browser and is not covered by the HTTP seam. - Before C1 is done, a tester on the dev environment creates a personal treatment step with - failure inputs, adds it to an assessment, sees the copied values, changes them, saves, - reopens, and confirms the personal treatment step itself is unchanged. + Before C1 is done again, a tester on the dev environment creates a personal treatment step + with all three failure inputs (including a number of parallel units above 1), adds it to + an assessment, sees the copied values, changes them, saves, reopens, and confirms the + personal treatment step itself is unchanged. - The existing test suite passes unchanged. ## Acceptance criteria 1. Each treatment step in the configurator, for guests and registered users, shows failure - frequency and failure duration with their units, starting at 0 and 30. + frequency, failure duration and number of parallel units with their labels, starting at + 0, 30 and 1. 2. The validation rules above reject invalid values with a message on the configurator, the guest result and the personal treatment step form, and accept every valid - combination, including the D4 cases. -3. Failure values survive save, reopen and edit of an assessment, per step. -4. Personal treatment steps store failure values, the configurator copies them into an - added step, and the copy can be changed independently (manual check on the dev + combination, including the D4 cases and any whole number of parallel units of 1 or more. +3. All three failure values survive save, reopen and edit of an assessment, per step. +4. Personal treatment steps store all three failure values, the configurator copies them + into an added step, and the copy can be changed independently (manual check on the dev environment). -5. Assessments and personal treatment steps that existed before C1 read back failure - frequency 0 and failure duration 30. -6. Results and export packages are the same as before C1 for every assessment. +5. Assessments and personal treatment steps that existed before a failure input read back + its default: failure frequency 0, failure duration 30, 1 parallel unit. +6. Results and export packages are the same as before C1 for every assessment, whatever its + failure inputs. 7. All HTTP-level tests listed above pass, and the existing suite passes unchanged. Quality goals from `docs/vision.md`, one criterion each: | Quality goal | Criterion in C1 | |---|---| -| Input validity | Every failure input shows its unit, and a value outside 0–365 days per year or 1–1440 whole minutes, or a frequency above 0 on a step without a positive LRV, is rejected with a message (criteria 1–2). | -| Reproducibility | Reopening a saved assessment returns exactly the stored failure values, and assessments from before C1 keep their results, since they read back frequency 0 (criteria 3, 5, 6). | +| Input validity | Every failure input shows its unit or meaning in its label; a value outside 0–365 days per year or 1–1440 whole minutes, a number of parallel units that is not a whole number of 1 or more, or a frequency above 0 on a step without a positive LRV is rejected with a message (criteria 1–2). | +| Reproducibility | Reopening a saved assessment returns exactly the stored failure values, and assessments from before C1 or before the number of parallel units keep their results, since they read back the defaults (criteria 3, 5, 6). | | Export stability | The export package of any assessment is unchanged by C1 (criterion 6); adding failure values to it is C3's job. | | Privacy of registered-user data | A personal treatment step and its failure values appear only in its owner's list; a test requests the list as another user and finds nothing of it (criterion 7). | +The number of parallel units has a lower bound but no upper bound (D7), although the Input +validity goal asks for a plausible range. That is intended: a mistyped large value can only +raise the failure LRV towards log10 of the number of units, at most to the step's normal +LRV (ADR-0002), so it cannot produce an implausible result. ADR-0002 is not reopened. + Not relevant to C1: *Scientific correctness* (the calculation is untouched; C2 and C5) and -*Traceability* (showing failure inputs in the export package and report is C3). +*Traceability* (showing failure inputs in the export package is C3). ## Out of Scope - Any effect of failure inputs on the calculation, the result page or the reference-level - exceedance (C2). -- Failure inputs in the export package and the HTML report (C3). -- Explanations of failure frequency, failure duration and the two cases beyond field - labels and units: FAQ, guided tour, Sphinx documentation (C4). + exceedance, including the failure LRV and the daily LRV (C2, ADR-0002). +- Failure inputs in the export package (C3). The HTML report shows no inputs and does not + get failure inputs either (decided 2026-09-25). +- Explanations beyond field labels and units: what a parallel unit is, that the failure + duration affects both cases, that units failing together are entered as 1 unit, and the + rest of the FAQ, guided tour and Sphinx documentation (C4). - Benchmark scenarios (C5). -- Failure data for bundled treatment steps. -- Partial loss of removal (not planned). +- Failure data for bundled treatment steps, including a number of parallel units. +- Units of one step that fail together; such a step is entered with 1 unit (D7). +- A failure frequency per unit; it stays per step (D7). +- Marking treatment steps as non-technical measures; every step that accepts failure + inputs gets the same three fields (D7). - Hiding the failure fields in production; D5 keeps C1 out of `main` instead. - A browser test setup; the one browser-only behaviour is checked manually. - The naming follow-ups in `docs/domain-model-open-questions.md` and the deferred UI @@ -222,9 +293,12 @@ Not relevant to C1: *Scientific correctness* (the calculation is untouched; C2 a ## Further Notes -- Until C2 is done, failure inputs have no effect on results. That is acceptable only - because of D5: C1 stays out of `main` and production until the release. -- While exploring: saving a personal treatment step checks the name for uniqueness against - the user's personal *exposures*, not their personal treatment steps. That is an existing - bug outside this Change; it is noted here so it is not mistaken for C1 behaviour. +- Until C2 is reworked, the number of parallel units has no effect on results, and C2's + current code still uses the rules ADR-0002 replaces. That is acceptable only because of + D5: C1–C3 stay out of `main` and production until the release. +- The default failure duration stays 30 minutes (roadmap, "Rework of C1–C3"). Since + ADR-0002 it affects worst-case too; C4 explains this, the configurator label does not. +- Saving a personal treatment step still checks the name for uniqueness against the user's + personal *exposures*, not their personal treatment steps. That is an existing bug outside + this Change; it is noted here so it is not mistaken for C1 behaviour. - The open roadmap decisions D1–D3 do not affect C1. From f9e4abda07c51abdb183e3df1520f066ff7f6a32 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 16 Oct 2026 15:55:34 +0000 Subject: [PATCH 23/43] docs: add C1 tickets 14-15 for D7, number of parallel units [skip ci] Co-Authored-By: Claude Opus 5.5 --- .../14-c1-parallel-units-in-configurator.md | 24 +++++++++++++++++++ ...-personal-treatment-step-parallel-units.md | 24 +++++++++++++++++++ 2 files changed, 48 insertions(+) create mode 100644 docs/tickets/14-c1-parallel-units-in-configurator.md create mode 100644 docs/tickets/15-c1-personal-treatment-step-parallel-units.md diff --git a/docs/tickets/14-c1-parallel-units-in-configurator.md b/docs/tickets/14-c1-parallel-units-in-configurator.md new file mode 100644 index 0000000..c1ff209 --- /dev/null +++ b/docs/tickets/14-c1-parallel-units-in-configurator.md @@ -0,0 +1,24 @@ +# 14: Number of parallel units on a treatment step in the configurator: enter, check, save, reopen, edit + +**What to build:** Every treatment step in the assessment configurator gets a third failure input next to failure frequency and failure duration: the **number of parallel units**, starting at 1. Guests and registered users see and use it. Registered users can save an assessment with it, and when they reopen or edit the assessment they see the same value again, for each step. The value does not change the result yet; the failure LRV follows in C2 (ADR-0002). + +The shared validation rule that both treatment step forms use is extended with the new input, not duplicated. A number of parallel units is a whole number of 1 or more, with no upper limit (D7). + +Delivered on branch `c1-failure-inputs` as part of the C1 pull request (#28). The pull request is tested on the dev environment again and not merged before the release (D5). + +Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.md`), Change C1 in `docs/failure_roadmap.md`, roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue: not created yet. + +**Blocked by:** None (can start immediately) + +**Status:** ready-for-agent + +- [ ] A schema migration adds the number of parallel units to the treatment steps of assessments. Existing treatment steps get 1. +- [ ] Each treatment step card in the configurator shows the field "Number of parallel units" below the LRV rows, next to failure frequency and failure duration, starting at 1. It is always shown, also when the failure frequency is 0. Guests see the same field. +- [ ] A bundled treatment step added to an assessment starts at 1 parallel unit. +- [ ] The shared rule rejects a number of parallel units of 0, a negative number or a fraction such as 1.5 with a message that names the field and its range, on the saved configurator and on the guest result. 1, 2 and a large value such as 1000 are accepted. +- [ ] Saving an assessment and reopening it returns the same number of parallel units per step. Editing and saving again stores the new value, and values stay with their step when other steps are reordered or removed. +- [ ] An assessment saved before this Ticket reopens with 1 parallel unit, failure frequency 0 and failure duration 30, and its result is unchanged. +- [ ] The result and the export package of an assessment are identical with and without failure inputs, including a number of parallel units above 1. +- [ ] HTTP-level tests through Django's test client cover the cases above, extending the existing C1 failure input tests. The existing test suite passes unchanged. + +Quality goals (`docs/vision.md`): Input validity (label and checked range), Reproducibility (saved values and old assessments read back unchanged), Export stability (export package unchanged by C1). diff --git a/docs/tickets/15-c1-personal-treatment-step-parallel-units.md b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md new file mode 100644 index 0000000..24c61b1 --- /dev/null +++ b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md @@ -0,0 +1,24 @@ +# 15: Personal treatment steps store the number of parallel units and the configurator copies it in + +**What to build:** A registered user can store a number of parallel units on a personal treatment step, next to its failure frequency and failure duration, with the same rule as in the configurator. When they add the step to an assessment, its number of parallel units is copied in together with its other failure inputs and its LRVs. The copy can then be changed in that one assessment without changing the personal treatment step. + +Delivered on branch `c1-failure-inputs` and completes the D7 rework of the C1 pull request (#28). The pull request is tested on the dev environment again and not merged before the release (D5). + +Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.md`), roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue: not created yet. + +**Blocked by:** 14 + +**Status:** ready-for-agent + +- [ ] A schema migration adds the number of parallel units to personal treatment steps. Existing personal treatment steps get 1. +- [ ] The personal treatment step form shows "Number of parallel units", starting at 1, next to the two other failure fields, and applies the same shared rule as the configurator. Invalid input is rejected and the message is shown in the form. +- [ ] A personal treatment step saved with a number of parallel units appears with it in its owner's list of personal treatment steps. Another user's list does not contain it. +- [ ] Adding a personal treatment step to an assessment copies its number of parallel units into the new step, together with its failure frequency, failure duration and LRVs. Bundled steps still start at 1. +- [ ] Changing the copied value in an assessment and saving leaves the personal treatment step unchanged. +- [ ] HTTP-level tests cover the form rule, personal treatment steps from before this Ticket reading back 1, and owner-only visibility. The existing test suite passes unchanged. +- [ ] Manual check on the dev environment, announced to the team first: + - Create a personal treatment step with all three failure inputs, including a number of parallel units above 1, and add it to an assessment. + - See the copied values, change them, save and reopen. + - Confirm that the personal treatment step itself is unchanged. + +Quality goals (`docs/vision.md`): Input validity (the same rule as the configurator), Privacy of registered-user data (the step and its failure inputs appear only in its owner's list). From 039fcf30415962d562593a3fb3b70b38e2fc32c8 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 16 Oct 2026 16:03:30 +0000 Subject: [PATCH 24/43] docs: link C1 tickets 14-15 to issues #36 and #37 [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/review-2026-10-dev-scenarios.md | 3 ++- docs/tickets/14-c1-parallel-units-in-configurator.md | 2 +- docs/tickets/15-c1-personal-treatment-step-parallel-units.md | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/review-2026-10-dev-scenarios.md b/docs/review-2026-10-dev-scenarios.md index 67b9bdb..601e5c4 100644 --- a/docs/review-2026-10-dev-scenarios.md +++ b/docs/review-2026-10-dev-scenarios.md @@ -135,4 +135,5 @@ Answers (2026-10-16): | Ticket | Finding | Change | Branch | |--------|---------|--------|--------| -| | | | | +| 14 (#36) Number of parallel units on a treatment step in the configurator | F2 (D7) | C1 | `c1-failure-inputs` | +| 15 (#37) Personal treatment steps store the number of parallel units | F2 (D7) | C1 | `c1-failure-inputs` | diff --git a/docs/tickets/14-c1-parallel-units-in-configurator.md b/docs/tickets/14-c1-parallel-units-in-configurator.md index c1ff209..898b55e 100644 --- a/docs/tickets/14-c1-parallel-units-in-configurator.md +++ b/docs/tickets/14-c1-parallel-units-in-configurator.md @@ -6,7 +6,7 @@ The shared validation rule that both treatment step forms use is extended with t Delivered on branch `c1-failure-inputs` as part of the C1 pull request (#28). The pull request is tested on the dev environment again and not merged before the release (D5). -Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.md`), Change C1 in `docs/failure_roadmap.md`, roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue: not created yet. +Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.md`), Change C1 in `docs/failure_roadmap.md`, roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue #36. **Blocked by:** None (can start immediately) diff --git a/docs/tickets/15-c1-personal-treatment-step-parallel-units.md b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md index 24c61b1..2f08033 100644 --- a/docs/tickets/15-c1-personal-treatment-step-parallel-units.md +++ b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md @@ -4,9 +4,9 @@ Delivered on branch `c1-failure-inputs` and completes the D7 rework of the C1 pull request (#28). The pull request is tested on the dev environment again and not merged before the release (D5). -Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.md`), roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue: not created yet. +Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.md`), roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue #37. -**Blocked by:** 14 +**Blocked by:** 14 (#36) **Status:** ready-for-agent From 64a68a59a77e94bc92434b00a309a8a9bebb4033 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 16 Oct 2026 16:03:57 +0000 Subject: [PATCH 25/43] docs: record the D6/D7 rework and C1 tickets 14-15 in the progress file [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/progress.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/progress.md b/docs/progress.md index a5bb425..4762ca9 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -1,6 +1,7 @@ # Progress: treatment failure (C1–C5) -Status: 2026-09-29. Work of 2026-09-24 to 2026-09-28 on the Changes in `docs/failure_roadmap.md`. +Status: 2026-10-16. Work of 2026-09-24 to 2026-09-28 on the Changes in `docs/failure_roadmap.md`, +and the rework after the dev review of October 2026 (D6, D7). Nothing is merged into `main` (roadmap decision D5): each Change is a draft pull request that targets `main` only so that the pipeline deploys it to dev. The production jobs run only for `main` itself. @@ -9,9 +10,9 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs | Change | Branch (based on) | Pull request | Tickets | State | |---|---|---|---|---| -| C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`) | Built, tested manually on dev | -| C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | Built, tested manually on dev | -| C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open | +| C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`); #36, #37 (`14`, `15`) | Reopened for D7: spec revised, tickets 14–15 ready | +| C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | Built; reopened for D6/D7, spec and tickets not revised yet | +| C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open; reopened for D7 (units column), ticket not written yet | | C4 Explain treatment failure | not created yet (will be based on C2) | — | #32, #33 (`12`, `13`) | Tickets ready | | C5 Failure benchmark | — | — | — | Waits for decisions D1–D3 | @@ -22,6 +23,12 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs - **Review notes:** ticket 09 records a review of the C2 commits against the Spec, ADR-0001 and the code; tickets 10 and 11 record a review of the C3 commits. - **D4 wording:** `CONTEXT.md`, the roadmap and the vision now say that worst-case loses every *positive* LRV of a failing step, as the code does. +**Dev review of October 2026** (`docs/review-2026-10-dev-scenarios.md`, merged into `main` as #35): +- **D6:** best-case and worst-case use one rule on a failure day, the daily LRV of each failing step (ADR-0002). It replaces the worst-case full-day loss and the overlap rules built in #25 and #26. +- **D7:** a third failure input, the number of parallel units; a failure stops one unit (ADR-0002). +- **Rework:** in new tickets on the existing pull requests (roadmap, "Rework of C1–C3"). `main` was merged into C1–C3 on 2026-10-16 (docs only). +- **C1:** spec revised for D7 (`e9d5ce7`), tickets 14 (#36) and 15 (#37) written. + ## What each Change delivers ### C1 Failure inputs (#28) From e547ca1dcaafb641cf11acd5653576f405b411ef Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 16 Oct 2026 16:16:24 +0000 Subject: [PATCH 26/43] feat: number of parallel units on treatment steps in the configurator (#36) Ticket 14 (D7): each treatment step of an assessment has a number of parallel units, a whole number of 1 or more with no upper limit, default 1. The configurator shows it next to failure frequency and failure duration, for guests and registered users; the shared rule rejects values below 1 and fractions with one message. A migration gives existing treatment steps 1 unit. Results and export are unchanged; the calculation follows in C2. Personal treatment steps follow in ticket 15 (#37). Co-Authored-By: Claude Opus 5.5 --- .../14-c1-parallel-units-in-configurator.md | 16 ++-- qmra/risk_assessment/forms.py | 15 +++- .../0013_treatment_parallel_units.py | 18 ++++ qmra/risk_assessment/models.py | 1 + .../tests/test_failure_inputs.py | 84 +++++++++++++++---- .../tests/test_risk_assessment_form.py | 2 + 6 files changed, 110 insertions(+), 26 deletions(-) create mode 100644 qmra/risk_assessment/migrations/0013_treatment_parallel_units.py diff --git a/docs/tickets/14-c1-parallel-units-in-configurator.md b/docs/tickets/14-c1-parallel-units-in-configurator.md index 898b55e..340f459 100644 --- a/docs/tickets/14-c1-parallel-units-in-configurator.md +++ b/docs/tickets/14-c1-parallel-units-in-configurator.md @@ -12,13 +12,13 @@ Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.m **Status:** ready-for-agent -- [ ] A schema migration adds the number of parallel units to the treatment steps of assessments. Existing treatment steps get 1. -- [ ] Each treatment step card in the configurator shows the field "Number of parallel units" below the LRV rows, next to failure frequency and failure duration, starting at 1. It is always shown, also when the failure frequency is 0. Guests see the same field. -- [ ] A bundled treatment step added to an assessment starts at 1 parallel unit. -- [ ] The shared rule rejects a number of parallel units of 0, a negative number or a fraction such as 1.5 with a message that names the field and its range, on the saved configurator and on the guest result. 1, 2 and a large value such as 1000 are accepted. -- [ ] Saving an assessment and reopening it returns the same number of parallel units per step. Editing and saving again stores the new value, and values stay with their step when other steps are reordered or removed. -- [ ] An assessment saved before this Ticket reopens with 1 parallel unit, failure frequency 0 and failure duration 30, and its result is unchanged. -- [ ] The result and the export package of an assessment are identical with and without failure inputs, including a number of parallel units above 1. -- [ ] HTTP-level tests through Django's test client cover the cases above, extending the existing C1 failure input tests. The existing test suite passes unchanged. +- [x] A schema migration adds the number of parallel units to the treatment steps of assessments. Existing treatment steps get 1. +- [x] Each treatment step card in the configurator shows the field "Number of parallel units" below the LRV rows, next to failure frequency and failure duration, starting at 1. It is always shown, also when the failure frequency is 0. Guests see the same field. +- [x] A bundled treatment step added to an assessment starts at 1 parallel unit. +- [x] The shared rule rejects a number of parallel units of 0, a negative number or a fraction such as 1.5 with a message that names the field and its range, on the saved configurator and on the guest result. 1, 2 and a large value such as 1000 are accepted. +- [x] Saving an assessment and reopening it returns the same number of parallel units per step. Editing and saving again stores the new value, and values stay with their step when other steps are reordered or removed. +- [x] An assessment saved before this Ticket reopens with 1 parallel unit, failure frequency 0 and failure duration 30, and its result is unchanged. +- [x] The result and the export package of an assessment are identical with and without failure inputs, including a number of parallel units above 1. +- [x] HTTP-level tests through Django's test client cover the cases above, extending the existing C1 failure input tests. The existing test suite passes unchanged. Quality goals (`docs/vision.md`): Input validity (label and checked range), Reproducibility (saved values and old assessments read back unchanged), Export stability (export package unchanged by C1). diff --git a/qmra/risk_assessment/forms.py b/qmra/risk_assessment/forms.py index d08f4df..a648fc7 100644 --- a/qmra/risk_assessment/forms.py +++ b/qmra/risk_assessment/forms.py @@ -23,11 +23,12 @@ def _zero_if_none(x): return x if x is not None else 0 "failure_frequency": (0, 365, "days per year"), "failure_duration": (1, 1440, "minutes"), } +PARALLEL_UNITS_MESSAGE = "number of parallel units must be a whole number of 1 or more" def check_treatment_step(form, cleaned_data): """Minimum LRV at most maximum LRV per pathogen group, failure inputs within their ranges, - and a failure frequency above 0 only for a step with a positive LRV. + at least one parallel unit, and a failure frequency above 0 only for a step with a positive LRV. Shared by the configurator and the personal treatment step form. Whole minutes are enforced by the failure duration's field type; failure inputs a form does not have are skipped. @@ -39,6 +40,10 @@ def check_treatment_step(form, cleaned_data): value = cleaned_data.get(field) if value is not None and not low <= value <= high: form.add_error(field, f"{field.replace('_', ' ')} must be between {low} and {high} {unit}") + # no upper limit: more units only raise the LRV during a failure (D7) + parallel_units = cleaned_data.get("parallel_units") + if parallel_units is not None and parallel_units < 1: + form.add_error("parallel_units", PARALLEL_UNITS_MESSAGE) # a failure removes only positive LRVs, so a step without one cannot fail (D4) has_positive_lrv = any(_zero_if_none(cleaned_data.get(field)) > 0 for pair in LRV_FIELDS for field in pair) if _zero_if_none(cleaned_data.get("failure_frequency")) > 0 and not has_positive_lrv: @@ -184,6 +189,7 @@ class Meta: "protozoa_max", "failure_frequency", "failure_duration", + "parallel_units", ] def __init__(self, *args, **kwargs): @@ -202,8 +208,11 @@ def __init__(self, *args, **kwargs): self.fields['protozoa_max'].label = "" self.fields['failure_frequency'].label = "Failure frequency (days per year)" self.fields['failure_duration'].label = "Failure duration (minutes)" - # leave whole minutes to the server check: the browser blocks hidden step cards without a message + self.fields['parallel_units'].label = "Number of parallel units" + # leave whole numbers to the server check: the browser blocks hidden step cards without a message self.fields['failure_duration'].widget.attrs['step'] = "any" + self.fields['parallel_units'].widget.attrs['step'] = "any" + self.fields['parallel_units'].error_messages['invalid'] = PARALLEL_UNITS_MESSAGE label_style = "class='text-muted text-center w-100' style='margin-top: .4em;'" self.helper.layout = Layout( Field("name", css_class="disabled-input d-none"), @@ -216,7 +225,7 @@ def __init__(self, *args, **kwargs): Column("viruses_min"), Column("viruses_max")), Row(Column(HTML(f"")), Column("protozoa_min"), Column("protozoa_max")), - Row(Column("failure_frequency"), Column("failure_duration")), + Row(Column("failure_frequency"), Column("failure_duration"), Column("parallel_units")), # Row(Column("DELETE")) ) diff --git a/qmra/risk_assessment/migrations/0013_treatment_parallel_units.py b/qmra/risk_assessment/migrations/0013_treatment_parallel_units.py new file mode 100644 index 0000000..92657c9 --- /dev/null +++ b/qmra/risk_assessment/migrations/0013_treatment_parallel_units.py @@ -0,0 +1,18 @@ +# Generated by Django 5.0.14 on 2026-10-16 16:11 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('risk_assessment', '0012_usertreatment_failure_inputs'), + ] + + operations = [ + migrations.AddField( + model_name='treatment', + name='parallel_units', + field=models.IntegerField(default=1), + ), + ] diff --git a/qmra/risk_assessment/models.py b/qmra/risk_assessment/models.py index eaeb936..70a6918 100644 --- a/qmra/risk_assessment/models.py +++ b/qmra/risk_assessment/models.py @@ -37,6 +37,7 @@ class Treatment(models.Model): protozoa_max = models.FloatField(blank=True, null=True) failure_frequency = models.FloatField(default=0) # failure days per year failure_duration = models.IntegerField(default=30) # minutes + parallel_units = models.IntegerField(default=1) # a failure event stops one of them (D7) @classmethod def from_default(cls, default: QMRATreatment, risk_assessment): diff --git a/qmra/risk_assessment/tests/test_failure_inputs.py b/qmra/risk_assessment/tests/test_failure_inputs.py index bf1a1a5..5e5a02b 100644 --- a/qmra/risk_assessment/tests/test_failure_inputs.py +++ b/qmra/risk_assessment/tests/test_failure_inputs.py @@ -81,7 +81,8 @@ def failure_inputs_per_step(data: dict) -> list[dict]: return [ dict(name=data[f"treatments-{i}-name"], failure_frequency=float(data[f"treatments-{i}-failure_frequency"]), - failure_duration=int(data[f"treatments-{i}-failure_duration"])) + failure_duration=int(data[f"treatments-{i}-failure_duration"]), + parallel_units=int(data[f"treatments-{i}-parallel_units"])) for i in range(int(data["treatments-TOTAL_FORMS"])) ] @@ -99,6 +100,7 @@ def new_configurator(client) -> dict: return data +PARALLEL_UNITS_MESSAGE = "number of parallel units must be a whole number of 1 or more" LRVS = dict(bacteria_min=1, bacteria_max=2, viruses_min=1, viruses_max=2, protozoa_min=1, protozoa_max=2) @@ -140,6 +142,13 @@ def test_new_step_card_shows_failure_fields_with_units_and_defaults(self): assert_that(float(data["treatments-__prefix__-failure_frequency"])).is_equal_to(0) assert_that(int(data["treatments-__prefix__-failure_duration"])).is_equal_to(30) + def test_new_step_card_shows_number_of_parallel_units_starting_at_1(self): + response = self.client.get(reverse("assessment")) + + assert_that(response.content.decode()).contains("Number of parallel units") + data = configurator_data(response) + assert_that(int(data["treatments-__prefix__-parallel_units"])).is_equal_to(1) + def test_bundled_step_starts_at_defaults(self): data = new_configurator(self.client) add_step(data, "Primary treatment", **LRVS) @@ -147,27 +156,31 @@ def test_bundled_step_starts_at_defaults(self): assessment = self.save_new(data) assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ - dict(name="Primary treatment", failure_frequency=0, failure_duration=30) + dict(name="Primary treatment", failure_frequency=0, failure_duration=30, parallel_units=1) ]) def test_failure_inputs_survive_save_and_reopen_per_step(self): data = new_configurator(self.client) add_step(data, "Primary treatment", **LRVS, failure_frequency="0.5", failure_duration="15") - add_step(data, "Primary treatment", **LRVS, failure_frequency="12", failure_duration="600") - add_step(data, "Slow sand filtration", **LRVS, failure_frequency="365", failure_duration="1440") + add_step(data, "Primary treatment", **LRVS, failure_frequency="12", failure_duration="600", + parallel_units="4") + add_step(data, "Slow sand filtration", **LRVS, failure_frequency="365", failure_duration="1440", + parallel_units="2") assessment = self.save_new(data) assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ - dict(name="Primary treatment", failure_frequency=0.5, failure_duration=15), - dict(name="Primary treatment", failure_frequency=12, failure_duration=600), - dict(name="Slow sand filtration", failure_frequency=365, failure_duration=1440), + dict(name="Primary treatment", failure_frequency=0.5, failure_duration=15, parallel_units=1), + dict(name="Primary treatment", failure_frequency=12, failure_duration=600, parallel_units=4), + dict(name="Slow sand filtration", failure_frequency=365, failure_duration=1440, parallel_units=2), ]) def test_editing_stores_new_values_and_values_stay_with_their_step(self): data = new_configurator(self.client) - add_step(data, "Primary treatment", **LRVS, failure_frequency="1", failure_duration="10") - add_step(data, "Slow sand filtration", **LRVS, failure_frequency="2", failure_duration="20") + add_step(data, "Primary treatment", **LRVS, failure_frequency="1", failure_duration="10", + parallel_units="3") + add_step(data, "Slow sand filtration", **LRVS, failure_frequency="2", failure_duration="20", + parallel_units="2") add_step(data, "Primary treatment", **LRVS, failure_frequency="3", failure_duration="30") assessment = self.save_new(data) @@ -175,12 +188,13 @@ def test_editing_stores_new_values_and_values_stay_with_their_step(self): data["treatments-0-DELETE"] = "on" data["treatments-2-failure_frequency"] = "4.5" data["treatments-2-failure_duration"] = "45" + data["treatments-2-parallel_units"] = "5" response = self.post_configurator(data, assessment) assert_that(response.status_code).is_equal_to(302) assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ - dict(name="Slow sand filtration", failure_frequency=2, failure_duration=20), - dict(name="Primary treatment", failure_frequency=4.5, failure_duration=45), + dict(name="Slow sand filtration", failure_frequency=2, failure_duration=20, parallel_units=2), + dict(name="Primary treatment", failure_frequency=4.5, failure_duration=45, parallel_units=5), ]) def test_out_of_range_or_fractional_values_are_rejected_with_a_message(self): @@ -216,6 +230,28 @@ def test_range_bounds_are_accepted(self): assert_that(response.status_code).is_equal_to(302) + def test_number_of_parallel_units_below_1_or_fractional_is_rejected_with_a_message(self): + for units in ["0", "-1", "1.5"]: + with self.subTest(units=units): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **LRVS, parallel_units=units) + + response = self.post_configurator(data) + + assert_that(response.status_code).is_equal_to(200) + assert_that(field_errors(response, "treatments-0-parallel_units")).is_equal_to([PARALLEL_UNITS_MESSAGE]) + assert_that(RiskAssessment.objects.filter(user=self.user).count()).is_equal_to(0) + + def test_any_whole_number_of_parallel_units_of_1_or_more_is_accepted(self): + for units in ["1", "2", "1000"]: + with self.subTest(units=units): + data = new_configurator(self.client) + add_step(data, "Primary treatment", **LRVS, parallel_units=units) + + response = self.post_configurator(data) + + assert_that(response.status_code).is_equal_to(302) + def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): assessment = RiskAssessment.objects.create(user=self.user, events_per_year=365, volume_per_event=1) for pathogen in ["Rotavirus", "Campylobacter jejuni", "Cryptosporidium parvum"]: @@ -223,7 +259,7 @@ def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): Treatment.objects.create(risk_assessment=assessment, name="Primary treatment", **LRVS) assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ - dict(name="Primary treatment", failure_frequency=0, failure_duration=30) + dict(name="Primary treatment", failure_frequency=0, failure_duration=30, parallel_units=1) ]) def test_result_and_export_do_not_change_with_failure_inputs(self): @@ -238,9 +274,12 @@ def test_result_and_export_do_not_change_with_failure_inputs(self): data["treatments-0-failure_frequency"] = "20" data["treatments-0-failure_duration"] = "120" data["treatments-1-failure_frequency"] = "365" + data["treatments-1-parallel_units"] = "4" self.post_configurator(data, assessment) - assert_that(failure_inputs_per_step(self.reopen(assessment))[0]["failure_frequency"]).is_equal_to(20) + steps = failure_inputs_per_step(self.reopen(assessment)) + assert_that(steps[0]["failure_frequency"]).is_equal_to(20) + assert_that(steps[1]["parallel_units"]).is_equal_to(4) assert_that(self.results(assessment)).is_equal_to(results_before) assert_that(self.export(assessment)).is_equal_to(export_before) @@ -302,7 +341,19 @@ def test_guest_sees_failure_fields(self): response = self.client.get(reverse("assessment")) assert_that(response.content.decode()).contains( - "Failure frequency (days per year)", "Failure duration (minutes)") + "Failure frequency (days per year)", "Failure duration (minutes)", "Number of parallel units") + + def test_guest_step_card_starts_at_1_parallel_unit(self): + data = configurator_data(self.client.get(reverse("assessment"))) + + assert_that(int(data["treatments-__prefix__-parallel_units"])).is_equal_to(1) + + def test_guest_result_accepts_any_whole_number_of_parallel_units_of_1_or_more(self): + for units in ["1", "2", "1000"]: + with self.subTest(units=units): + response = self.guest_result(parallel_units=units) + + assert_that(response.status_code).is_equal_to(200) def test_guest_result_with_failure_inputs_is_shown_and_not_stored(self): response = self.guest_result(failure_frequency="12", failure_duration="60") @@ -317,6 +368,9 @@ def test_guest_result_sends_back_messages_of_invalid_inputs(self): "failure frequency must be between 0 and 365 days per year"), (dict(failure_duration="2.5"), "treatments-0-failure_duration", "Enter a whole number."), (dict(bacteria_min="3", bacteria_max="1"), "treatments-0-bacteria_min", "min. must be less than max"), + (dict(parallel_units="0"), "treatments-0-parallel_units", PARALLEL_UNITS_MESSAGE), + (dict(parallel_units="-1"), "treatments-0-parallel_units", PARALLEL_UNITS_MESSAGE), + (dict(parallel_units="1.5"), "treatments-0-parallel_units", PARALLEL_UNITS_MESSAGE), ] for step_values, field, message in cases: with self.subTest(field=field, message=message): @@ -437,5 +491,5 @@ def test_changing_the_copy_in_an_assessment_leaves_the_personal_step_unchanged(s assert_that(failure_inputs_per_step(configurator_data( self.client.get(reverse("assessment", args=[assessment.id]))))).is_equal_to([ - dict(name="Plant UV", failure_frequency=10, failure_duration=15)]) + dict(name="Plant UV", failure_frequency=10, failure_duration=15, parallel_units=1)]) assert_that(self.failure_inputs("Plant UV")).is_equal_to((2.5, 90)) diff --git a/qmra/risk_assessment/tests/test_risk_assessment_form.py b/qmra/risk_assessment/tests/test_risk_assessment_form.py index d1ae84b..e3d6fd6 100644 --- a/qmra/risk_assessment/tests/test_risk_assessment_form.py +++ b/qmra/risk_assessment/tests/test_risk_assessment_form.py @@ -90,6 +90,7 @@ def test_that_negative_are_allowed(self): protozoa_max=-1, failure_frequency=0, failure_duration=30, + parallel_units=1, ) given_form = TreatmentForm(data=data) # ugly hack to work around dynamic choices... @@ -109,6 +110,7 @@ def test_that_min_needs_to_be_less_than_max(self): protozoa_max=0, failure_frequency=0, failure_duration=30, + parallel_units=1, ) for prfx in ["bacteria", "viruses", "protozoa"]: mn = prfx + "_min" From f19a791ab0c1edc8fa88593b4f7420d87b0367f3 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Fri, 16 Oct 2026 16:46:39 +0000 Subject: [PATCH 27/43] feat: personal treatment steps store the number of parallel units (#37) Ticket 15 (D7): a personal treatment step has a number of parallel units, default 1, checked by the same shared rule as the configurator. A migration gives existing personal treatment steps 1 unit. The owner's list includes the value, and the configurator copies it into a newly added step together with the other failure inputs and the LRVs; the copy can be changed without changing the personal treatment step. The manual check on dev is still open. Co-Authored-By: Claude Opus 5.5 --- ...-personal-treatment-step-parallel-units.md | 12 ++++---- .../0014_usertreatment_parallel_units.py | 18 +++++++++++ .../templates/treatments-form-js.html | 1 + .../tests/test_failure_inputs.py | 30 +++++++++++++------ qmra/risk_assessment/user_models.py | 8 +++-- 5 files changed, 52 insertions(+), 17 deletions(-) create mode 100644 qmra/risk_assessment/migrations/0014_usertreatment_parallel_units.py diff --git a/docs/tickets/15-c1-personal-treatment-step-parallel-units.md b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md index 2f08033..60a5bb8 100644 --- a/docs/tickets/15-c1-personal-treatment-step-parallel-units.md +++ b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md @@ -10,12 +10,12 @@ Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.m **Status:** ready-for-agent -- [ ] A schema migration adds the number of parallel units to personal treatment steps. Existing personal treatment steps get 1. -- [ ] The personal treatment step form shows "Number of parallel units", starting at 1, next to the two other failure fields, and applies the same shared rule as the configurator. Invalid input is rejected and the message is shown in the form. -- [ ] A personal treatment step saved with a number of parallel units appears with it in its owner's list of personal treatment steps. Another user's list does not contain it. -- [ ] Adding a personal treatment step to an assessment copies its number of parallel units into the new step, together with its failure frequency, failure duration and LRVs. Bundled steps still start at 1. -- [ ] Changing the copied value in an assessment and saving leaves the personal treatment step unchanged. -- [ ] HTTP-level tests cover the form rule, personal treatment steps from before this Ticket reading back 1, and owner-only visibility. The existing test suite passes unchanged. +- [x] A schema migration adds the number of parallel units to personal treatment steps. Existing personal treatment steps get 1. +- [x] The personal treatment step form shows "Number of parallel units", starting at 1, next to the two other failure fields, and applies the same shared rule as the configurator. Invalid input is rejected and the message is shown in the form. +- [x] A personal treatment step saved with a number of parallel units appears with it in its owner's list of personal treatment steps. Another user's list does not contain it. +- [x] Adding a personal treatment step to an assessment copies its number of parallel units into the new step, together with its failure frequency, failure duration and LRVs. Bundled steps still start at 1. +- [x] Changing the copied value in an assessment and saving leaves the personal treatment step unchanged. +- [x] HTTP-level tests cover the form rule, personal treatment steps from before this Ticket reading back 1, and owner-only visibility. The existing test suite passes unchanged. - [ ] Manual check on the dev environment, announced to the team first: - Create a personal treatment step with all three failure inputs, including a number of parallel units above 1, and add it to an assessment. - See the copied values, change them, save and reopen. diff --git a/qmra/risk_assessment/migrations/0014_usertreatment_parallel_units.py b/qmra/risk_assessment/migrations/0014_usertreatment_parallel_units.py new file mode 100644 index 0000000..7b23805 --- /dev/null +++ b/qmra/risk_assessment/migrations/0014_usertreatment_parallel_units.py @@ -0,0 +1,18 @@ +# Generated by Django 5.0.14 on 2026-10-16 16:43 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('risk_assessment', '0013_treatment_parallel_units'), + ] + + operations = [ + migrations.AddField( + model_name='usertreatment', + name='parallel_units', + field=models.IntegerField(default=1), + ), + ] diff --git a/qmra/risk_assessment/templates/treatments-form-js.html b/qmra/risk_assessment/templates/treatments-form-js.html index e3bafa5..cd26e91 100644 --- a/qmra/risk_assessment/templates/treatments-form-js.html +++ b/qmra/risk_assessment/templates/treatments-form-js.html @@ -21,6 +21,7 @@ protozoa_max: { selector: "input[id*='protozoa_max']", getter: (elem) => elem.value, setter: (elem, x) => { elem.value = x } }, failure_frequency: { selector: "input[id*='failure_frequency']", getter: (elem) => elem.value, setter: (elem, x) => { elem.value = x } }, failure_duration: { selector: "input[id*='failure_duration']", getter: (elem) => elem.value, setter: (elem, x) => { elem.value = x } }, + parallel_units: { selector: "input[id*='parallel_units']", getter: (elem) => elem.value, setter: (elem, x) => { elem.value = x } }, } constructor(div) { this.$form = div diff --git a/qmra/risk_assessment/tests/test_failure_inputs.py b/qmra/risk_assessment/tests/test_failure_inputs.py index 5e5a02b..ca9cc46 100644 --- a/qmra/risk_assessment/tests/test_failure_inputs.py +++ b/qmra/risk_assessment/tests/test_failure_inputs.py @@ -414,7 +414,7 @@ def personal_step_form(response) -> str: class TestPersonalTreatmentStepFailureInputs(LoggedInTestCase): def create_personal_step(self, name="Plant UV", **values): - data = dict(name=name, **LRVS, failure_frequency="0", failure_duration="30") + data = dict(name=name, **LRVS, failure_frequency="0", failure_duration="30", parallel_units="1") data.update(values) return self.client.post(reverse("treatment"), data, HTTP_REFERER=reverse("assessment")) @@ -423,7 +423,7 @@ def personal_steps(self, client=None) -> dict: def failure_inputs(self, name: str) -> tuple: step = self.personal_steps()[name] - return step["failure_frequency"], step["failure_duration"] + return step["failure_frequency"], step["failure_duration"], step["parallel_units"] def test_form_shows_failure_fields_with_units_and_defaults(self): form = personal_step_form(self.client.get(reverse("assessment"))) @@ -432,11 +432,17 @@ def test_form_shows_failure_fields_with_units_and_defaults(self): assert_that(form).matches(r'name="failure_frequency" value="0"') assert_that(form).matches(r'name="failure_duration" value="30"') + def test_form_shows_number_of_parallel_units_starting_at_1(self): + form = personal_step_form(self.client.get(reverse("assessment"))) + + assert_that(form).contains("Number of parallel units") + assert_that(form).matches(r'name="parallel_units" value="1"') + def test_failure_inputs_appear_in_the_owners_list(self): - response = self.create_personal_step(failure_frequency="2.5", failure_duration="90") + response = self.create_personal_step(failure_frequency="2.5", failure_duration="90", parallel_units="3") assert_that(response.status_code).is_equal_to(302) - assert_that(self.failure_inputs("Plant UV")).is_equal_to((2.5, 90)) + assert_that(self.failure_inputs("Plant UV")).is_equal_to((2.5, 90, 3)) def test_other_users_do_not_see_the_personal_step(self): self.create_personal_step(failure_frequency="2.5", failure_duration="90") @@ -454,6 +460,10 @@ def test_invalid_values_are_rejected_with_a_message(self): (dict(failure_duration=""), "This field is required."), (dict(NO_LRVS, failure_frequency="1"), D4_MESSAGE), (dict(bacteria_min="3", bacteria_max="1"), "min. must be less than max"), + (dict(parallel_units="0"), PARALLEL_UNITS_MESSAGE), + (dict(parallel_units="-1"), PARALLEL_UNITS_MESSAGE), + (dict(parallel_units="1.5"), PARALLEL_UNITS_MESSAGE), + (dict(parallel_units=""), "This field is required."), ] for values, message in cases: with self.subTest(message=message, values=values): @@ -467,6 +477,8 @@ def test_valid_values_including_d4_cases_are_accepted(self): cases = { "bounds low": dict(failure_frequency="0", failure_duration="1"), "bounds high": dict(failure_frequency="365", failure_duration="1440"), + "2 parallel units": dict(parallel_units="2"), + "1000 parallel units": dict(parallel_units="1000"), "no positive LRV, frequency 0": dict(NO_LRVS, failure_frequency="0"), **{case: dict(lrvs, failure_frequency="1") for case, lrvs in D4_ACCEPTED.items()}, } @@ -480,16 +492,16 @@ def test_valid_values_including_d4_cases_are_accepted(self): def test_personal_step_from_before_failure_inputs_offers_defaults(self): UserTreatment.objects.create(user=self.user, name="Old step", **LRVS) - assert_that(self.failure_inputs("Old step")).is_equal_to((0, 30)) + assert_that(self.failure_inputs("Old step")).is_equal_to((0, 30, 1)) def test_changing_the_copy_in_an_assessment_leaves_the_personal_step_unchanged(self): - self.create_personal_step(failure_frequency="2.5", failure_duration="90") + self.create_personal_step(failure_frequency="2.5", failure_duration="90", parallel_units="3") data = new_configurator(self.client) - add_step(data, "Plant UV", **LRVS, failure_frequency="10", failure_duration="15") + add_step(data, "Plant UV", **LRVS, failure_frequency="10", failure_duration="15", parallel_units="5") assessment = self.save_new(data) assert_that(failure_inputs_per_step(configurator_data( self.client.get(reverse("assessment", args=[assessment.id]))))).is_equal_to([ - dict(name="Plant UV", failure_frequency=10, failure_duration=15, parallel_units=1)]) - assert_that(self.failure_inputs("Plant UV")).is_equal_to((2.5, 90)) + dict(name="Plant UV", failure_frequency=10, failure_duration=15, parallel_units=5)]) + assert_that(self.failure_inputs("Plant UV")).is_equal_to((2.5, 90, 3)) diff --git a/qmra/risk_assessment/user_models.py b/qmra/risk_assessment/user_models.py index 9e3891e..f969160 100644 --- a/qmra/risk_assessment/user_models.py +++ b/qmra/risk_assessment/user_models.py @@ -6,7 +6,7 @@ from crispy_forms.helper import FormHelper from crispy_forms.layout import Layout, Field, Row, Column, HTML, Submit -from qmra.risk_assessment.forms import check_treatment_step +from qmra.risk_assessment.forms import check_treatment_step, PARALLEL_UNITS_MESSAGE from qmra.user.models import User @@ -140,6 +140,7 @@ class UserTreatment(models.Model): protozoa_max = models.FloatField(blank=True, null=True) failure_frequency = models.FloatField(default=0) # failure days per year failure_duration = models.IntegerField(default=30) # minutes + parallel_units = models.IntegerField(default=1) # a failure event stops one of them (D7) class UserTreatmentForm(forms.ModelForm): @@ -155,6 +156,7 @@ class Meta: "protozoa_max", "failure_frequency", "failure_duration", + "parallel_units", ] def __init__(self, *args, **kwargs): @@ -173,6 +175,8 @@ def __init__(self, *args, **kwargs): self.fields['protozoa_max'].label = "" self.fields['failure_frequency'].label = "Failure frequency (days per year)" self.fields['failure_duration'].label = "Failure duration (minutes)" + self.fields['parallel_units'].label = "Number of parallel units" + self.fields['parallel_units'].error_messages['invalid'] = PARALLEL_UNITS_MESSAGE label_style = "class='text-muted text-center w-100' style='margin-top: .4em;'" self.helper.layout = Layout( Field("name"), @@ -185,7 +189,7 @@ def __init__(self, *args, **kwargs): Column("viruses_min"), Column("viruses_max")), Row(Column(HTML(f"")), Column("protozoa_min"), Column("protozoa_max")), - Row(Column("failure_frequency"), Column("failure_duration")), + Row(Column("failure_frequency"), Column("failure_duration"), Column("parallel_units")), Submit('submit', 'Submit') ) From 96243a7c4b48e33f360cde79669b86eacfc6cc84 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:07:30 +0000 Subject: [PATCH 28/43] docs: tick the manual check on dev of C1 ticket 15 (#37) [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/tickets/15-c1-personal-treatment-step-parallel-units.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tickets/15-c1-personal-treatment-step-parallel-units.md b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md index 60a5bb8..879848e 100644 --- a/docs/tickets/15-c1-personal-treatment-step-parallel-units.md +++ b/docs/tickets/15-c1-personal-treatment-step-parallel-units.md @@ -16,7 +16,7 @@ Source: Spec C1 Failure inputs, revision for D7 (`docs/specs/c1-failure-inputs.m - [x] Adding a personal treatment step to an assessment copies its number of parallel units into the new step, together with its failure frequency, failure duration and LRVs. Bundled steps still start at 1. - [x] Changing the copied value in an assessment and saving leaves the personal treatment step unchanged. - [x] HTTP-level tests cover the form rule, personal treatment steps from before this Ticket reading back 1, and owner-only visibility. The existing test suite passes unchanged. -- [ ] Manual check on the dev environment, announced to the team first: +- [x] Manual check on the dev environment, announced to the team first (2026-10-17, passed): - Create a personal treatment step with all three failure inputs, including a number of parallel units above 1, and add it to an assessment. - See the copied values, change them, save and reopen. - Confirm that the personal treatment step itself is unchanged. From d97e27578667109c2014db0c4488daa05f36c091 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:09:16 +0000 Subject: [PATCH 29/43] docs: C1 tickets 14-15 built and checked on dev [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/progress.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/progress.md b/docs/progress.md index 4762ca9..201f939 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -1,6 +1,6 @@ # Progress: treatment failure (C1–C5) -Status: 2026-10-16. Work of 2026-09-24 to 2026-09-28 on the Changes in `docs/failure_roadmap.md`, +Status: 2026-10-17. Work of 2026-09-24 to 2026-09-28 on the Changes in `docs/failure_roadmap.md`, and the rework after the dev review of October 2026 (D6, D7). Nothing is merged into `main` (roadmap decision D5): each Change is a draft pull request that targets `main` only so that the pipeline deploys it to dev. The production jobs run only for @@ -10,13 +10,13 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs | Change | Branch (based on) | Pull request | Tickets | State | |---|---|---|---|---| -| C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`); #36, #37 (`14`, `15`) | Reopened for D7: spec revised, tickets 14–15 ready | +| C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`); #36, #37 (`14`, `15`) | D7 rework built (tickets 14–15), tested manually on dev | | C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | Built; reopened for D6/D7, spec and tickets not revised yet | | C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open; reopened for D7 (units column), ticket not written yet | | C4 Explain treatment failure | not created yet (will be based on C2) | — | #32, #33 (`12`, `13`) | Tickets ready | | C5 Failure benchmark | — | — | — | Waits for decisions D1–D3 | -**Dev** (https://dev.qmra.org) runs C1 + C2 + C3 (PR #34, commit `27b5268`) since 2026-09-25. +**Dev** (https://dev.qmra.org) runs C1 (PR #28, commit `f19a791`) since 2026-10-16. C2 and C3 are not on dev until they are pushed again. **Review of 2026-09-28** (by a teammate): - **Statuses:** tickets 04–11 are set to "review". @@ -27,7 +27,10 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs - **D6:** best-case and worst-case use one rule on a failure day, the daily LRV of each failing step (ADR-0002). It replaces the worst-case full-day loss and the overlap rules built in #25 and #26. - **D7:** a third failure input, the number of parallel units; a failure stops one unit (ADR-0002). - **Rework:** in new tickets on the existing pull requests (roadmap, "Rework of C1–C3"). `main` was merged into C1–C3 on 2026-10-16 (docs only). -- **C1:** spec revised for D7 (`e9d5ce7`), tickets 14 (#36) and 15 (#37) written. +- **C1:** spec revised for D7 (`e9d5ce7`), tickets 14 (#36) and 15 (#37) written and built: + - **Configurator (#36, `e547ca1`):** each treatment step has a number of parallel units (a whole number of 1 or more, no upper limit, default 1) next to failure frequency and failure duration, for guests and registered users; the shared rule rejects values below 1 and fractions with one message; existing assessments get 1. + - **Personal treatment steps (#37, `f19a791`):** they store the number of parallel units under the same rule, and the configurator copies it into a newly added step; existing personal treatment steps get 1. + - **Manual check on dev** passed on 2026-10-17. C1 was merged forward into C2 and C3; their tests pass. ## What each Change delivers From 3196dc4c71893ee2573fdbe202c5c32583a78dd2 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:17:01 +0000 Subject: [PATCH 30/43] docs: revise the C2 spec for D6 and D7, daily LRV and parallel units [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/specs/c2-failure-calculation.md | 340 ++++++++++++++++----------- 1 file changed, 201 insertions(+), 139 deletions(-) diff --git a/docs/specs/c2-failure-calculation.md b/docs/specs/c2-failure-calculation.md index a484904..b8f46a8 100644 --- a/docs/specs/c2-failure-calculation.md +++ b/docs/specs/c2-failure-calculation.md @@ -1,33 +1,53 @@ # Spec C2: Failure calculation -Status: draft, 2026-09-24. Change C2 in `docs/failure_roadmap.md` (Large). Spec issue: #___ -Sources: `docs/vision.md` (MVP item 3 and 4), `CONTEXT.md`, ADR-0001, the decisions in -`docs/QMRA_Failure_Approach.md`, roadmap decisions D4 and D5. Builds on C1 -(`docs/specs/c1-failure-inputs.md`). +Status: draft, 2026-09-24, revised 2026-10-17 for D6 (one daily LRV rule for both cases) +and D7 (number of parallel units). Change C2 in `docs/failure_roadmap.md` (Large). Spec +issue: #___ +Sources: `docs/vision.md` (MVP item 3 and 4), `CONTEXT.md`, ADR-0001, ADR-0002, the +decisions in `docs/QMRA_Failure_Approach.md`, roadmap decisions D4–D7. Builds on C1 +(`docs/specs/c1-failure-inputs.md`, revised for D7). Worked example: +`docs/failure-daily-lrv-example.md`. + +The first version is built and was tested on the dev environment (tickets 07–09). It drew +failure days per exposure event, lost a failing step's positive minimum LRVs for the whole +day in worst-case, and applied the mixed-water assumption with overlap rules in best-case. +This revision keeps the failure-day draws and replaces the rule for the LRV on a failure +day: both cases now use the daily LRV of each failing step, and a step with several +parallel units loses only the share of the failing unit (ADR-0002). Requirements that are +new or changed in this revision are marked *(D6)* or *(D7)*. ## Problem Statement -After C1, users can enter a failure frequency and a failure duration for each treatment -step, but the result ignores them: an assessment with a UV unit failing 10 days per year -gives the same annual probability of infection as one where it never fails. A utility -operator therefore still cannot see how failures of their treatment steps affect the -yearly risk, whether the reference level is still met, or how two failure scenarios -compare. +After C1, users enter a failure frequency, a failure duration and a number of parallel +units for each treatment step. A utility operator wants the result to show how these +failures affect the yearly risk, whether the reference level is still met, and how two +failure scenarios compare. + +The first version of C2 answers this with two different rules. Worst-case ignores the +failure duration, so a one-minute and a full-day failure give the same worst-case risk. +Best-case needs special overlap rules for failures of several steps on one day. Neither +case knows parallel units, so every failure is counted as a complete loss of the step's +positive LRVs, although a step often has several units that do not fail at the same time. +The review on dev in October 2026 found these results implausible (findings F1 and F2 in +`docs/review-2026-10-dev-scenarios.md`). ## Solution -The risk calculation takes treatment failures into account as decided in ADR-0001. In the -Monte Carlo simulation, every simulated exposure event falls on a failure day of each -treatment step with probability failure frequency / 365. On a failure day: +The risk calculation takes treatment failures into account as decided in ADR-0001 and +ADR-0002. In the Monte Carlo simulation, every simulated exposure event falls on a failure +day of each treatment step with probability failure frequency / 365 (ADR-0001, unchanged). +On a failure day, best-case and worst-case use the same rule *(D6)*: -- **worst-case** calculates the event with the failing steps' positive LRVs removed for the - whole day, whatever the failure duration; -- **best-case** applies the mixed-water assumption: the consumed water is a mixture of water - treated during the failure events and water treated in normal operation, in proportion to - the failure durations. Failure events that fit into one day do not overlap; two steps - longer than a day in total overlap only by the minutes beyond the day; three or more steps - longer than a day in total count like worst-case for that day. Only steps that lose LRV - for the pathogen group count. +- each failing step gets a **daily LRV**: the water consumed that day is a mixture of water + treated during the failure event and water treated in normal operation, in proportion to + the failure duration (the mixed-water assumption), averaged by concentration; +- during the failure event the step has its **failure LRV**: with N parallel units, one + unit stops and the others keep working, so 1/N of the water is not treated *(D7)*. With + one unit the failure LRV is 0; +- the treatment train's LRV that day is the sum of the failing steps' daily LRVs and the + other steps' normal LRVs, so combined failures need no overlap rules *(D6)*; +- the only difference between the two cases is the LRV each step starts from: its maximum + LRV in best-case and its minimum LRV in worst-case. LRVs of 0 or below are never affected (D4). The result page, the reference-level exceedance, the saved results and the result table in the export package all show the @@ -44,96 +64,108 @@ failure frequency is 0 gives exactly the same results as before C2. 3. As a utility operator, I want both risk measures (annual probability of infection and DALYs per person per year) to include failures, so that I can judge the health burden as well as the infection risk. -4. As a user, I want worst-case to assume that all water consumed on a failure day was - treated during the failure, so that worst-case stays the cautious estimate. -5. As a user, I want best-case to assume mixed water on a failure day, weighted by the - failure duration, so that best-case reflects storage that dilutes failure water. -6. As a user, I want a longer failure duration to raise the best-case risk and leave the - worst-case risk unchanged, so that the two cases behave as documented. -7. As a user, I want a higher failure frequency never to lower the risk, so that the results - make sense when I try different values. -8. As a user, I want worst-case risk never to be below best-case risk, so that the two cases - keep their meaning with failures. -9. As a user, I want the number of failure days to vary between simulated years, so that - the risk distribution shows years with more or fewer failures, as real years do. -10. As a user with an exposure of few events per year, I want a low failure frequency to +4. As a user, I want best-case and worst-case to use the same failure rule, so that the + difference between the two cases comes only from the minimum and maximum LRVs. *(D6)* +5. As a user, I want the failure duration to count in both cases, so that a one-minute + failure and a full-day failure give different results in worst-case too. *(D6)* +6. As a user, I want a longer failure duration never to lower the risk, in either case, so + that the results make sense when I try different values. *(D6)* +7. As a user, I want a failure lasting the whole day (1,440 minutes) of a step with one + unit to remove that step's positive LRVs for the day, so that the extreme case matches + the obvious expectation. *(D6)* +8. As a user, I want a step with several parallel units to lose only the share of the + failing unit, so that a failure of one of four filters is not counted as a failure of + all four. *(D7)* +9. As a user, I want the LRV of a step during a failure to follow from the share of + untreated water, so that two units keep about 0.3 log and ten units about 1 log during a + failure, whatever the step's normal LRV. *(D7)* +10. As a user, I want more parallel units never to raise the risk, so that describing my + plant more precisely never makes it look worse. *(D7)* +11. As a user, I want a step with one parallel unit to lose all its positive LRVs during the + failure, so that the default behaves as before. *(D7)* +12. As a user, I want a higher failure frequency never to lower the risk, so that the + results make sense when I try different values. +13. As a user, I want worst-case risk never to be below best-case risk, so that the two + cases keep their meaning with failures. +14. As a user, I want the number of failure days to vary between simulated years, so that + the risk distribution shows years with more or fewer failures, as real years do. +15. As a user with an exposure of few events per year, I want a low failure frequency to show up as an upper tail of the risk distribution, so that I see the rare bad years. -11. As a user with several failing steps, I want failures of different steps on the same +16. As a user with several failing steps, I want failures of different steps on the same day to be combined, so that combined failures are not missed. -12. As a user, I want a combined failure in best-case to assume that failure events do not - overlap when they fit into one day, so that best-case stays the favourable assumption. -13. As a user, I want two failure events longer than a day in total to overlap only by the - minutes beyond the day, so that best-case stays favourable without a jump at 24 hours. - When three or more steps fail on the same day for longer than a day in total, I accept - that best-case counts that day like worst-case, since so much downtime on one day is a - rare exception. -14. As a user, I want a failure to remove only the positive LRVs of a step, so that a step +17. As a user, I want combined failures to be calculated as if each failure event happened + at a random time of day, independently of the others, so that the result is neither + the most favourable nor the least favourable arrangement. *(D6)* +18. As a user, I want two steps failing on the same day for a long time each to be + calculated without a special rule for durations that add up to more than a day, so + that there is no jump in the result at 24 hours. *(D6)* +19. As a user, I want a failure to remove only the positive LRVs of a step, so that a step with regrowth (a negative LRV) in one pathogen group does not improve during a failure. -15. As a user, I want a pathogen group whose LRV at a step is 0 or empty to be unaffected by +20. As a user, I want a pathogen group whose LRV at a step is 0 or empty to be unaffected by that step's failure, so that failures only matter where the step removes pathogens. -16. As a user, I want every failure frequency at 0 to give exactly the results I got before, +21. As a user, I want a step with a minimum LRV of 0 and a positive maximum LRV to affect + only best-case when it fails, so that each case loses only what it counted. +22. As a user, I want every failure frequency at 0 to give exactly the results I got before, so that existing assessments and comparisons keep their numbers. -17. As a registered user, I want saved assessments to store the result including failures, +23. As a registered user, I want saved assessments to store the result including failures, so that the saved-assessments page and the assessment comparison show it. -18. As a user comparing two saved assessments that differ only in their failure inputs, I - want the comparison to show different infection risks, so that I see what the failures - cost. -19. As a registered user, I want the export package's result table to contain the result +24. As a user comparing two saved assessments that differ only in their failure inputs, for + example only in the number of parallel units, I want the comparison to show different + infection risks, so that I see what the failures cost. +25. As a registered user, I want the export package's result table to contain the result including failures, so that my report matches the result page. -20. As a guest, I want the unsaved result to include failures in the same way, so that the +26. As a guest, I want the unsaved result to include failures in the same way, so that the result does not depend on having an account. -21. As a user, I want the same assessment to give the same result every time it is +27. As a user, I want the same assessment to give the same result every time it is calculated, so that I can reproduce and discuss a result. -22. As a reviewer, I want the calculation to follow ADR-0001 and the documented decisions, - so that the result can be checked against an independent calculation (C5). -23. As a user, I want the result page to look as before, so that I do not need to learn a +28. As a reviewer, I want the calculation to follow ADR-0001, ADR-0002 and the worked + example, so that the result can be checked against an independent calculation (C5). +29. As a user, I want the result page to look as before, so that I do not need to learn a new layout. -24. As a user, I want the calculation with failures to finish in about the same time as +30. As a user, I want the calculation with failures to finish in about the same time as before, so that running an assessment stays quick. -25. As a developer of C5, I want the calculation to be deterministic for a given scenario, +31. As a developer of C5, I want the calculation to be deterministic for a given scenario, so that benchmark comparisons are stable. -26. As a developer of C4, I want the calculation's rules to be stated in the Spec in plain +32. As a developer of C4, I want the calculation's rules to be stated in the Spec in plain terms, so that the explanations for users can be written from it. ## Implementation Decisions - **Only the calculation module changes.** Its public entry point keeps its interface: it receives an assessment, its inflow concentrations and its treatment steps, and returns one - result per reference pathogen. It now reads each treatment step's failure frequency and - failure duration (stored by C1). The result record, the result page, saving and the export - are not changed; they show whatever the calculation returns. -- **What a failure removes (D4).** For each treatment step and pathogen group, worst-case - can lose the step's minimum LRV and best-case its maximum LRV, each only if it is above 0. - LRVs of 0, below 0 or empty stay in place during a failure. -- **Failure days per exposure event (ADR-0001).** For every simulated exposure event of every - simulated year, and for every treatment step with failure frequency above 0, the step - fails that day if a uniform random number is below failure frequency / 365. The same - random numbers are used for both cases and all reference pathogens. As a consequence, - raising a failure frequency only ever adds failure days, and worst-case and best-case see - the same failure days. -- **Worst-case on a failure day.** The event's LRV is the worst-case LRV of the train minus - the removable minimum LRVs of all steps failing that day. The failure duration is not - used. -- **Best-case on a failure day (mixed-water assumption).** The day is split into time - segments by which failing steps are down. The event's LRV is −log₁₀ of the time-weighted - average of 10^(−LRV of each segment), where a segment's LRV is the best-case LRV of the - train minus the removable maximum LRVs of the steps down in it (`CONTEXT.md`, *Combined - failure*): - - failure events whose durations fit into 1,440 minutes do not overlap; for a single - failing step this equals Eq. 5 of the failure approach; - - two failure events longer than a day in total overlap only by the minutes beyond the day; - - three or more failure events longer than a day in total count like worst-case: all these - steps are down for the whole day and lose their removable maximum LRVs. Only steps with a - removable maximum LRV for the pathogen group count, since a failure changes nothing where - a step removes nothing (D4); so a day can count like worst-case for one reference - pathogen and not for another. This replaces - the lowest-concentration arrangement decided on 2026-09-24, which needed a numerical - solver for a rare exception (decided 2026-09-25). - Every rule has a closed formula, so no solver is needed. + result per reference pathogen. It reads each treatment step's failure frequency, failure + duration and number of parallel units (stored by C1). The result record, the result page, + saving and the export are not changed; they show whatever the calculation returns. +- **What a failure can affect (D4).** For each treatment step and pathogen group, worst-case + starts from the step's minimum LRV and best-case from its maximum LRV. Only an LRV above + 0 is affected by a failure; LRVs of 0, below 0 or empty stay in place. +- **Failure days per exposure event (ADR-0001, unchanged).** For every simulated exposure + event of every simulated year, and for every treatment step with failure frequency above + 0, the step fails that day if a uniform random number is below failure frequency / 365. + The same random numbers are used for both cases and all reference pathogens. Each step + draws from its own stream, keyed by its position in the train, from a fixed seed. As a + consequence, raising a failure frequency only ever adds failure days, and worst-case and + best-case see the same failure days. +- **Failure LRV of a step (D7, ADR-0002).** For a positive LRV of a step with N parallel + units, the failure LRV is −log₁₀((N − 1)/N · 10^(−LRV) + 1/N): the untreated and the + treated water are mixed by concentration. It is 0 for one unit and at most log₁₀ N. +- **Daily LRV of a failing step (D6, ADR-0002).** With d = failure duration / 1,440, the + daily LRV is −log₁₀(d · 10^(−failure LRV) + (1 − d) · 10^(−LRV)), averaged by + concentration, never by LRV. LRV is the step's maximum LRV in best-case and its minimum + LRV in worst-case. +- **LRV of an exposure event on a failure day (D6).** The event's LRV is the case's LRV of + the train minus, for every step failing that day, the difference between that step's LRV + and its daily LRV. The two cases use the same rule. Combined failures need no further + rule: summing the steps' daily LRVs equals placing each failure event at a random time of + day, independently of the other steps. +- **Rules that are removed (D6).** The worst-case loss of the failing steps' positive + minimum LRVs for the whole day regardless of the duration, the best-case arrangement of + combined failures without overlap, the overlap by the minutes beyond a day, and the rule + that three or more steps longer than a day in total count like worst-case. - **Concentration sampling unchanged.** The sampled inflow concentrations and the choice of which sample each simulated event uses stay exactly as today, including the fixed seed. - The failure draws come from their own fixed-seed random generator. When all failure - frequencies are 0, no failure draws are made and the results are identical to today's. + When all failure frequencies are 0, no failure draws are made and the results are + identical to today's. - **Dose and yearly risk.** Each event's dose follows from its sampled concentration, its LRV (normal, or reduced as above) and the volume per event, and the yearly risk combines the events of a simulated year as today. The summary statistics and the reference-level @@ -141,14 +173,18 @@ failure frequency is 0 gives exactly the same results as before C2. - **Saved results.** Saving an assessment recalculates and stores its results as today. Results stored before C2 stay valid: every assessment in production has failure frequency 0 at the release, so its stored results equal what C2 calculates. -- **Speed.** When failures are present, the dose-response is evaluated per simulated event - instead of per concentration sample. The work per failure day depends only on which steps - fail, so repeated combinations can be reused. The calculation time for a drinking-water - scenario (365 events per year) with failures is measured on the dev environment and - recorded in the pull request; the vision sets no response-time target yet. +- **Speed.** The dose-response is evaluated once per distinct event LRV and concentration + sample, as in the first version. Each failing step contributes one fixed LRV difference + per case and pathogen group, so the number of distinct event LRVs is bounded by the + combinations of failing steps. The calculation time for a drinking-water scenario (365 + events per year) with failures is measured and recorded in the pull request; the first + version took 0.13–0.19 s locally against 0.07 s without failures. The vision sets no + response-time target yet. - **Unchanged warnings.** The warning for a maximum LRV above 6 is not changed. -- **Delivery.** C2 is built on C1's pull request, stays open and is tested on the dev - environment; it is not merged into `main` before the release (D5). +- **Delivery.** The rework is done in new tickets on the existing C2 pull request (#29), + which builds on C1's pull request (roadmap decision "Rework of C1–C3"). Tickets 08 and 09 + get a note naming the tickets that partly replace them. C2 stays open and is tested on the + dev environment again; it is not merged into `main` before the release (D5). ## Testing Decisions @@ -156,49 +192,70 @@ failure frequency is 0 gives exactly the same results as before C2. concentrations, treatment steps with failure inputs, exposure) and checks the returned results: mean, median, quartiles, minimum and maximum per reference pathogen, for both cases and both risk measures, and the reference-level exceedance. It does not inspect - intermediate arrays or random draws. + intermediate arrays or random draws. Expected values come from an independent source: + the results of a scenario without failures that has the expected constant LRV, the + worked example, or the failure LRV table of D7; never from the formula the code uses. + The worked example and the D7 table give LRVs rounded to two decimals, so comparisons + with them allow for that rounding. - **One seam: the calculation's public entry point**, called the way the existing calculation tests call it. The result page, saving and export only pass results through, - and C1's HTTP tests already show that failure inputs arrive at the calculation. C2 adds no - HTTP tests. + and C1's HTTP tests show that the three failure inputs arrive at the calculation. C2 adds + no HTTP tests. - **Behaviour covered at that seam** (internal checks; the independent benchmark is C5): - **No failures:** with every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. - - **Always failing, worst-case:** a step with failure frequency 365 gives the same - worst-case results as the scenario without that step's positive minimum LRVs. - - **Always failing, full day, best-case:** failure frequency 365 and duration 1,440 give - the same best-case results as the scenario without that step's positive maximum LRVs. - - **Always failing, part of the day, best-case:** failure frequency 365 and duration 60 - give the same best-case results as the scenario with the train's best-case LRV replaced - by the constant mixed LRV from Eq. 5, computed by hand in the test. - - **Combined failures:** two steps always failing with 900 + 900 minutes equal the - hand-computed constant LRV with 360 minutes of overlap; three steps always failing within - a day in total equal the hand-computed LRV without overlap; three steps always failing - with more than a day in total equal the scenario without their positive maximum LRVs; - a failing step without removal for one pathogen group does not count for that group. + - **Always failing, full day, one unit** *(D6)*: a step with failure frequency 365, + duration 1,440 and one unit gives the same results in both cases as the scenario + without that step's positive LRVs (daily LRV 0). + - **Always failing, part of the day** *(D6)*: failure frequency 365 and duration 60 give, + in both cases, the same results as a scenario whose train has the hand-computed constant + daily LRV; for the worked example's step with LRV 4 next to a step with LRV 3, the train + LRV of the day is 4.38. + - **Combined failures, worked example** *(D6)*: two steps with LRV 4 and 3, both always + failing for 60 minutes with one unit, give the same results as a train with constant + LRV 2.75; for 4 h, 12 h and 16 h each, 1.55, 0.60 and 0.35. These values come from + `docs/failure-daily-lrv-example.md`. + - **Average by concentration** *(D6)*: in the 60-minute combined case the result differs + clearly from a train with the plain LRV average of 6.71. + - **Parallel units** *(D7)*: a step with LRV 4 always failing for the whole day with 2, + 4 or 10 units gives the same results as a train in which that step has LRV 0.30, 0.60 + or 1.00 (the D7 table); with one unit, LRV 0. - **Monotonicity:** raising a failure frequency never lowers any mean; raising a failure - duration never lowers a best-case mean and leaves worst-case unchanged. + duration never lowers any mean, in either case *(D6)*; raising the number of parallel + units never raises any mean *(D7)*. - **Case order:** worst-case mean ≥ best-case mean for every pathogen and risk measure, with and without failures. - **D4:** a step with a negative LRV for one pathogen group and a positive LRV for another leaves the first group's results unchanged when it fails; a group with LRV 0 is - unaffected. + unaffected; a step with minimum LRV 0 and a positive maximum LRV changes only + best-case when it fails. - **Reproducibility:** calculating the same scenario twice gives identical results. +- **Tests of the first version that change.** The worst-case tests for a full-day loss + regardless of the duration, and for the duration not changing worst-case, are replaced + by the full-day and monotonicity checks above. The best-case tests for overlap beyond a + day, three steps within a day, three steps over a day counting like worst-case and three + steps crossing a day are removed with their rules. - **Prior art.** The existing calculation tests: a scenario built from bundled data and - passed to the calculation, and the regression test comparing fixed numbers. -- **Manual check on the dev environment.** Enter a scenario with failures in the - configurator, run it as a guest and as a registered user, and confirm the result page, - the saved-assessments page and the export's result table show the changed results. Record - the calculation time. -- The existing test suite passes unchanged. + passed to the calculation, the regression test comparing fixed numbers, and the C2 tests + that compare an always-failing scenario with a scenario of constant LRV. +- **Export.** C1's HTTP test that the export package keeps its files and columns with + failures, and the reference export test of C3, keep passing; only result values of + assessments with failures change. +- **Manual check on the dev environment.** Enter the worked example's two steps with + failures in the configurator, run it as a guest and as a registered user, change the + failure duration and the number of parallel units, and confirm that both cases react and + that the result page, the saved-assessments page and the export's result table show the + changed results. Record the calculation time. +- The existing test suite passes, apart from the first-version tests named above. ## Acceptance criteria -1. The calculation follows the rules above: failure days per exposure event, worst-case - full loss, best-case mixed-water assumption with the combined-failure rules above, and - only positive LRVs lost. +1. The calculation follows the rules above: failure days per exposure event, the failure + LRV from the number of parallel units, the daily LRV of each failing step in both cases, + summed over the train, and only positive LRVs affected. 2. With every failure frequency at 0, all results equal today's exactly. -3. All tests listed above pass, and the existing suite passes unchanged. +3. The results match the worked example and the D7 failure LRV table in the tests listed + above, and all other listed tests pass. 4. The result page, the reference-level exceedance, saved results, the assessment comparison and the export's result table show the result including failures, with unchanged layout (manual check on the dev environment). @@ -209,7 +266,7 @@ Quality goals from `docs/vision.md`, one criterion each: | Quality goal | Criterion in C2 | |---|---| -| Scientific correctness | The internal checks pass, including the closed-form cases (always failing steps compared with hand-computed LRVs) and the frequency-0 regression (criteria 2–3). Agreement with an independent calculation is C5. | +| Scientific correctness | The internal checks pass, with expected values from the worked example and the D7 table rather than from the code, and the frequency-0 regression holds (criteria 2–3). Agreement with an independent calculation is C5. | | Reproducibility | A scenario gives identical results on every calculation because the failure draws use their own fixed seed, and assessments without failures keep their exact results (criteria 2, 5). | | Export stability | The export package keeps its files and columns; only the result values of assessments with failures change (criterion 4). | @@ -218,27 +275,32 @@ data* (no change to access). ## Out of Scope -- Failure inputs, their validation and storage (C1). -- Failure inputs in the export package and the HTML report (C3). +- Failure inputs, their validation and storage, including the number of parallel units + (C1). +- Failure inputs in the export package, including a column for the number of parallel + units (C3). - Explanations for users in the FAQ, guided tour and documentation (C4). - Comparison with an independent calculation and the benchmark scenarios (C5); the tolerance (D1) is not needed here. +- A draw per exposure event of whether it falls inside the failure event, instead of the + average concentration (ADR-0002, Consequences). +- Units of one step that fail together, and a failure frequency per unit (D7). - Showing normal operation next to the result with failures (decided against). - DALYs or inputs in the assessment comparison (later). -- Partial loss of removal (not planned). - Recording the model version and simulation settings with results (reproducible assessment snapshots, later). - A response-time target (open question in the vision). ## Further Notes -- The rule for three or more failure events of one day in best-case changed during - implementation (#26, 2026-09-25). The lowest-concentration arrangement decided on - 2026-09-24 needed a linear program whose standard solver failed when the failing steps - lose about 16 LRV or more together. Since such days are a rare exception, they now count - like worst-case. `CONTEXT.md` (*Combined failure*) and decision 6 in - `docs/QMRA_Failure_Approach.md` are updated. -- No contradiction with ADR-0001 was found. ADR-0001 notes that two exposure events on the - same calendar day need not share a failure state; this Spec keeps that. -- Results with failures will differ from a hand calculation with Eq. 6 of the failure - approach, as ADR-0001 explains; they agree on average. +- No contradiction with an accepted ADR was found. ADR-0002 replaces ADR-0001's rule for the + LRV on a failure day; the per-event draw of failure days and the note that two exposure + events on the same calendar day need not share a failure state stay. +- With the default failure duration of 30 minutes, worst-case results with failures are + much less conservative than in the first version (ADR-0002, Consequences). This is + intended; C4 explains that the duration affects both cases. +- Results with failures still differ from a hand calculation with Eq. 6 of the failure + approach, as ADR-0001 explains, and now also in worst-case, which the source calculates + without the duration (ADR-0002). +- Assessments saved on the dev environment with the first version keep their stored + results until they are saved again. Production has no assessments with failures. From 933cc0ab2571dc551af54e2bc461b80aaad07991 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:21:20 +0000 Subject: [PATCH 31/43] docs: add C2 tickets 16-17 for D6 and D7; mark tickets 08 and 09 as partly replaced [skip ci] Co-Authored-By: Claude Opus 5.5 --- .../08-c2-failure-days-and-worst-case.md | 2 ++ docs/tickets/09-c2-best-case-mixed-water.md | 2 ++ docs/tickets/16-c2-daily-lrv-in-both-cases.md | 30 +++++++++++++++++++ .../17-c2-failure-lrv-from-parallel-units.md | 26 ++++++++++++++++ 4 files changed, 60 insertions(+) create mode 100644 docs/tickets/16-c2-daily-lrv-in-both-cases.md create mode 100644 docs/tickets/17-c2-failure-lrv-from-parallel-units.md diff --git a/docs/tickets/08-c2-failure-days-and-worst-case.md b/docs/tickets/08-c2-failure-days-and-worst-case.md index 070879b..1ed87e5 100644 --- a/docs/tickets/08-c2-failure-days-and-worst-case.md +++ b/docs/tickets/08-c2-failure-days-and-worst-case.md @@ -12,6 +12,8 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), AD **Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) +**Partly replaced (2026-10-17):** The worst-case loss for the whole day regardless of the failure duration is replaced by the daily LRV of Ticket 16 (D6, ADR-0002); the failure-day draws stay. + - [x] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. - [x] A step with failure frequency 365 gives the same worst-case results as the same scenario without that step's positive minimum LRVs. - [x] Raising a failure frequency never lowers any mean. Changing a failure duration leaves worst-case unchanged. diff --git a/docs/tickets/09-c2-best-case-mixed-water.md b/docs/tickets/09-c2-best-case-mixed-water.md index 5211614..bf42edf 100644 --- a/docs/tickets/09-c2-best-case-mixed-water.md +++ b/docs/tickets/09-c2-best-case-mixed-water.md @@ -17,6 +17,8 @@ Source: Spec C2 Failure calculation (`docs/specs/c2-failure-calculation.md`), AD **Status:** review (built in pull request #29; CI green on `a8a1972`; the pull request stays open until the release, D5) +**Partly replaced (2026-10-17):** The overlap rules for combined failures are replaced by the daily LRV of Ticket 16 (D6, ADR-0002), and the failure LRV of Ticket 17 replaces the full loss of a failing step (D7). + - [x] A step with failure frequency 365 and failure duration 1,440 gives the same best-case results as the same scenario without that step's positive maximum LRVs. - [x] A step with failure frequency 365 and failure duration 60 gives the same best-case results as the scenario with the train's best-case LRV replaced by the Eq. 5 mixed LRV, computed by hand in the test. - [x] Two steps always failing with 900 + 900 minutes equal the hand-computed result with 360 minutes of overlap. diff --git a/docs/tickets/16-c2-daily-lrv-in-both-cases.md b/docs/tickets/16-c2-daily-lrv-in-both-cases.md new file mode 100644 index 0000000..569ecdc --- /dev/null +++ b/docs/tickets/16-c2-daily-lrv-in-both-cases.md @@ -0,0 +1,30 @@ +# 16: Both cases use the daily LRV of each failing step + +**What to build:** On a failure day, best-case and worst-case use the same rule (D6, ADR-0002). Each failing treatment step gets a **daily LRV** under the mixed-water assumption: the water consumed that day is a mixture of water treated during the failure event and water treated in normal operation, in proportion to the failure duration, averaged by concentration and never by LRV. The step starts from its maximum LRV in best-case and from its minimum LRV in worst-case. The LRV of an exposure event on a failure day is the case's LRV of the train minus, for every step failing that day, the difference between its LRV and its daily LRV. Combined failures need no further rule. + +In this Ticket a failing step has LRV 0 during the failure event, which is the failure LRV of one parallel unit. The number of parallel units follows in Ticket 17. + +The rules of the first version are removed: the worst-case loss of the failing steps' positive minimum LRVs for the whole day regardless of the failure duration, the best-case arrangement of combined failures without overlap, the overlap by the minutes beyond a day, and the rule that three or more steps longer than a day in total count like worst-case. The failure-day draws per exposure event (ADR-0001), the concentration sampling and the fixed seeds stay as they are. LRVs of 0, below 0 or empty are not affected by a failure (D4). + +The result page, the reference-level exceedance, saved results and the export's result table show whatever the calculation returns. Their layout does not change. + +Delivered on branch `c2-failure-calculation` as part of the C2 pull request (#29). The pull request is not merged before the release (D5). + +Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-failure-calculation.md`), roadmap decision D6, ADR-0002, worked example `docs/failure-daily-lrv-example.md`. The Spec issue does not exist yet. GitHub issue: not created yet. + +**Blocked by:** None (can start immediately) + +**Status:** ready-for-agent + +- [ ] Both cases calculate a failing step's daily LRV from its failure duration, starting from the step's maximum LRV in best-case and its minimum LRV in worst-case, and sum the daily LRVs of all steps failing that day over the train. +- [ ] The worst-case full-day loss and the best-case overlap rules are removed from the calculation. +- [ ] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. +- [ ] A step always failing for 1,440 minutes gives the same results in both cases as the scenario without that step's positive LRVs. +- [ ] The results match the worked example (`docs/failure-daily-lrv-example.md`), allowing for its two-decimal rounding: one step with LRV 4 failing 60 minutes next to a step with LRV 3 gives the results of a train with constant LRV 4.38; both steps failing 60 minutes, 4 h, 12 h and 16 h each give those of 2.75, 1.55, 0.60 and 0.35. +- [ ] In the 60-minute combined case, the result differs clearly from a train with the plain LRV average of 6.71. +- [ ] Raising a failure duration never lowers any mean, in either case; raising a failure frequency never lowers any mean; worst-case mean is never below best-case mean. +- [ ] D4: a negative LRV of one pathogen group stays unchanged when the step fails, a group with LRV 0 is unaffected, and a step with minimum LRV 0 and a positive maximum LRV changes only best-case when it fails. +- [ ] The same scenario gives identical results on every calculation. +- [ ] The first-version tests for the worst-case full-day loss, the duration not changing worst-case, and the best-case overlap rules are replaced or removed as the Spec names them. All other tests pass. + +Quality goals (`docs/vision.md`): Scientific correctness (expected values from the worked example, not from the code), Reproducibility (fixed seeds, frequency 0 unchanged). diff --git a/docs/tickets/17-c2-failure-lrv-from-parallel-units.md b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md new file mode 100644 index 0000000..0c42c60 --- /dev/null +++ b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md @@ -0,0 +1,26 @@ +# 17: The failure LRV follows from the number of parallel units, and C2 is checked on dev + +**What to build:** A failing treatment step with N parallel units keeps the share of water treated by the other units during the failure event (D7, ADR-0002). For each positive LRV, its failure LRV is the LRV of the mixture of untreated water (1/N) and water treated with the step's LRV ((N − 1)/N), mixed by concentration. It is 0 for one unit, as in Ticket 16, and at most log₁₀ N. The daily LRV of Ticket 16 uses this failure LRV in both cases. Failure frequency stays per step, and LRVs of 0 or below are not affected (D4). + +This Ticket closes the D6/D7 rework of C2: it includes the manual check on the dev environment and the time measurement. + +Delivered on branch `c2-failure-calculation` as part of the C2 pull request (#29). The pull request is tested on the dev environment again and not merged before the release (D5). + +Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-failure-calculation.md`), roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue: not created yet. + +**Blocked by:** 16 + +**Status:** ready-for-agent + +- [ ] The calculation reads each treatment step's number of parallel units and uses the failure LRV in the daily LRV of both cases. +- [ ] A step with one parallel unit gives the same results as in Ticket 16. +- [ ] A step with LRV 4 always failing for the whole day with 2, 4 or 10 parallel units gives the same results as a train in which that step has LRV 0.30, 0.60 or 1.00 (the D7 table, allowing for its two-decimal rounding). +- [ ] Raising the number of parallel units never raises any mean. +- [ ] All tests pass, and the export package keeps its files and columns with failures. +- [ ] The calculation time of a drinking-water scenario (365 events per year) with failures is measured and recorded in the C2 pull request. +- [ ] Manual check on the dev environment, announced to the team first: + - Enter the worked example's two steps with failures and run them as a guest and as a registered user. + - Change the failure duration and the number of parallel units, and see both cases react. + - Confirm that the result page, the saved-assessments page and the export's result table show the changed results. + +Quality goals (`docs/vision.md`): Scientific correctness (expected values from the D7 table, not from the code), Export stability (files and columns unchanged). From 6377abb780832a72efc9d21c98bbc596e0e321c6 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:25:20 +0000 Subject: [PATCH 32/43] docs: link C2 tickets 16-17 to issues #38 and #39 [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/review-2026-10-dev-scenarios.md | 2 ++ docs/tickets/16-c2-daily-lrv-in-both-cases.md | 2 +- docs/tickets/17-c2-failure-lrv-from-parallel-units.md | 4 ++-- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/review-2026-10-dev-scenarios.md b/docs/review-2026-10-dev-scenarios.md index 601e5c4..a6a6950 100644 --- a/docs/review-2026-10-dev-scenarios.md +++ b/docs/review-2026-10-dev-scenarios.md @@ -137,3 +137,5 @@ Answers (2026-10-16): |--------|---------|--------|--------| | 14 (#36) Number of parallel units on a treatment step in the configurator | F2 (D7) | C1 | `c1-failure-inputs` | | 15 (#37) Personal treatment steps store the number of parallel units | F2 (D7) | C1 | `c1-failure-inputs` | +| 16 (#38) Both cases use the daily LRV of each failing step | F1 (D6) | C2 | `c2-failure-calculation` | +| 17 (#39) The failure LRV follows from the number of parallel units | F2 (D7) | C2 | `c2-failure-calculation` | diff --git a/docs/tickets/16-c2-daily-lrv-in-both-cases.md b/docs/tickets/16-c2-daily-lrv-in-both-cases.md index 569ecdc..cd5ecbc 100644 --- a/docs/tickets/16-c2-daily-lrv-in-both-cases.md +++ b/docs/tickets/16-c2-daily-lrv-in-both-cases.md @@ -10,7 +10,7 @@ The result page, the reference-level exceedance, saved results and the export's Delivered on branch `c2-failure-calculation` as part of the C2 pull request (#29). The pull request is not merged before the release (D5). -Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-failure-calculation.md`), roadmap decision D6, ADR-0002, worked example `docs/failure-daily-lrv-example.md`. The Spec issue does not exist yet. GitHub issue: not created yet. +Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-failure-calculation.md`), roadmap decision D6, ADR-0002, worked example `docs/failure-daily-lrv-example.md`. The Spec issue does not exist yet. GitHub issue #38. **Blocked by:** None (can start immediately) diff --git a/docs/tickets/17-c2-failure-lrv-from-parallel-units.md b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md index 0c42c60..fad67fe 100644 --- a/docs/tickets/17-c2-failure-lrv-from-parallel-units.md +++ b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md @@ -6,9 +6,9 @@ This Ticket closes the D6/D7 rework of C2: it includes the manual check on the d Delivered on branch `c2-failure-calculation` as part of the C2 pull request (#29). The pull request is tested on the dev environment again and not merged before the release (D5). -Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-failure-calculation.md`), roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue: not created yet. +Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-failure-calculation.md`), roadmap decision D7, ADR-0002. The Spec issue does not exist yet. GitHub issue #39. -**Blocked by:** 16 +**Blocked by:** 16 (#38) **Status:** ready-for-agent From 99ad33d6876cca7ebe3ea264fca0f04734532cf5 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:31:49 +0000 Subject: [PATCH 33/43] feat: both cases use the daily LRV of each failing step (#38) Ticket 16 (D6, ADR-0002): on a failure day, best-case and worst-case use the same rule. Each failing step gets its daily LRV under the mixed-water assumption, averaged by concentration, starting from its maximum LRV in best-case and its minimum LRV in worst-case; the failing steps' losses are summed over the train. A failing step has LRV 0 during the failure (one parallel unit); parallel units follow in ticket 17 (#39). Removes the worst-case full-day loss regardless of the duration and the best-case overlap rules, with their tests. New tests compare with the worked example (docs/failure-daily-lrv-example.md). Results without failures are unchanged. Co-Authored-By: Claude Opus 5.5 --- docs/tickets/16-c2-daily-lrv-in-both-cases.md | 20 +-- qmra/risk_assessment/risk.py | 84 ++++------- .../risk_assessment/tests/test_assess_risk.py | 138 +++++++++--------- 3 files changed, 104 insertions(+), 138 deletions(-) diff --git a/docs/tickets/16-c2-daily-lrv-in-both-cases.md b/docs/tickets/16-c2-daily-lrv-in-both-cases.md index cd5ecbc..7e6deca 100644 --- a/docs/tickets/16-c2-daily-lrv-in-both-cases.md +++ b/docs/tickets/16-c2-daily-lrv-in-both-cases.md @@ -16,15 +16,15 @@ Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-fail **Status:** ready-for-agent -- [ ] Both cases calculate a failing step's daily LRV from its failure duration, starting from the step's maximum LRV in best-case and its minimum LRV in worst-case, and sum the daily LRVs of all steps failing that day over the train. -- [ ] The worst-case full-day loss and the best-case overlap rules are removed from the calculation. -- [ ] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. -- [ ] A step always failing for 1,440 minutes gives the same results in both cases as the scenario without that step's positive LRVs. -- [ ] The results match the worked example (`docs/failure-daily-lrv-example.md`), allowing for its two-decimal rounding: one step with LRV 4 failing 60 minutes next to a step with LRV 3 gives the results of a train with constant LRV 4.38; both steps failing 60 minutes, 4 h, 12 h and 16 h each give those of 2.75, 1.55, 0.60 and 0.35. -- [ ] In the 60-minute combined case, the result differs clearly from a train with the plain LRV average of 6.71. -- [ ] Raising a failure duration never lowers any mean, in either case; raising a failure frequency never lowers any mean; worst-case mean is never below best-case mean. -- [ ] D4: a negative LRV of one pathogen group stays unchanged when the step fails, a group with LRV 0 is unaffected, and a step with minimum LRV 0 and a positive maximum LRV changes only best-case when it fails. -- [ ] The same scenario gives identical results on every calculation. -- [ ] The first-version tests for the worst-case full-day loss, the duration not changing worst-case, and the best-case overlap rules are replaced or removed as the Spec names them. All other tests pass. +- [x] Both cases calculate a failing step's daily LRV from its failure duration, starting from the step's maximum LRV in best-case and its minimum LRV in worst-case, and sum the daily LRVs of all steps failing that day over the train. +- [x] The worst-case full-day loss and the best-case overlap rules are removed from the calculation. +- [x] With every failure frequency at 0, all results equal today's exactly, including the existing regression test's numbers. +- [x] A step always failing for 1,440 minutes gives the same results in both cases as the scenario without that step's positive LRVs. +- [x] The results match the worked example (`docs/failure-daily-lrv-example.md`), allowing for its two-decimal rounding: one step with LRV 4 failing 60 minutes next to a step with LRV 3 gives the results of a train with constant LRV 4.38; both steps failing 60 minutes, 4 h, 12 h and 16 h each give those of 2.75, 1.55, 0.60 and 0.35. +- [x] In the 60-minute combined case, the result differs clearly from a train with the plain LRV average of 6.71. +- [x] Raising a failure duration never lowers any mean, in either case; raising a failure frequency never lowers any mean; worst-case mean is never below best-case mean. +- [x] D4: a negative LRV of one pathogen group stays unchanged when the step fails, a group with LRV 0 is unaffected, and a step with minimum LRV 0 and a positive maximum LRV changes only best-case when it fails. +- [x] The same scenario gives identical results on every calculation. +- [x] The first-version tests for the worst-case full-day loss, the duration not changing worst-case, and the best-case overlap rules are replaced or removed as the Spec names them. All other tests pass. Quality goals (`docs/vision.md`): Scientific correctness (expected values from the worked example, not from the code), Reproducibility (fixed seeds, frequency 0 unchanged). diff --git a/qmra/risk_assessment/risk.py b/qmra/risk_assessment/risk.py index 40258e0..505f00f 100644 --- a/qmra/risk_assessment/risk.py +++ b/qmra/risk_assessment/risk.py @@ -52,67 +52,35 @@ def draw_failing_events(treatments: list[Treatment], events_per_year: int, def failing_steps(treatments: list[Treatment], failing_events: list[np.ndarray | None], group: PathogenGroup, bound: str) -> list[tuple[np.ndarray, float, int]]: - """The steps that fail on some days and then lose LRV for this pathogen group, as (their failing events, LRV - lost, failure duration). A step whose minimum or maximum LRV is 0 or below loses nothing when it fails (D4).""" + """The steps that fail on some days and then lose LRV for this pathogen group, as (their failing events, their + minimum or maximum LRV, failure duration). A step whose LRV is 0 or below loses nothing when it fails (D4).""" steps = [] for t, failing in zip(treatments, failing_events): - lost = step_lrv(t, group, bound) - if failing is not None and lost > 0: - steps.append((failing, lost, t.failure_duration)) + lrv = step_lrv(t, group, bound) + if failing is not None and lrv > 0: + steps.append((failing, lrv, t.failure_duration)) return steps -def worst_case_event_lrvs(worst_case_lrv: float, treatments: list[Treatment], failing_events: list[np.ndarray | None], - group: PathogenGroup, shape: tuple) -> np.ndarray: - """Worst-case LRV of each exposure event: on a failure day the failing steps' positive minimum LRVs are lost for - the whole day, whatever the failure duration.""" - event_lrvs = np.full(shape, worst_case_lrv, dtype=float) - for failing, lost, _ in failing_steps(treatments, failing_events, group, "min"): - event_lrvs[failing] -= lost - return event_lrvs - - -def mixed_water_lrv_loss(failures: list[tuple[float, int]]) -> float: - """How much LRV the mixed water of a failure day loses in best-case (CONTEXT.md, Combined failure). +def daily_lrv_loss(lrv: float, failure_duration: int) -> float: + """How much LRV a failing step loses on a failure day under the mixed-water assumption (ADR-0002). - `failures` holds (LRV lost, failure duration in minutes) of each step that fails that day and loses LRV for the - pathogen group. The consumed water mixes water treated during the failure events and in normal operation, in - proportion to the failure durations. Failure events that fit into one day do not overlap; two longer than a day - in total overlap only by the minutes beyond the day. Three or more longer than a day in total count like - worst-case: all these steps are down for the whole day. + The consumed water mixes water treated during the failure event, with LRV 0, and water treated in normal + operation, in proportion to the failure duration. The mixture is averaged by concentration, never by LRV. """ - total_duration = sum(duration for _, duration in failures) - all_lost = sum(lost for lost, _ in failures) - if len(failures) >= 3 and total_duration > MINUTES_PER_DAY: - return all_lost - overlap = max(0, total_duration - MINUTES_PER_DAY) - segments = [(lost, duration - overlap) for lost, duration in failures] - if overlap: - segments.append((all_lost, overlap)) - normal_minutes = MINUTES_PER_DAY - sum(minutes for _, minutes in segments) - relative_concentration = normal_minutes + sum(minutes * 10 ** lost for lost, minutes in segments) - return np.log10(relative_concentration / MINUTES_PER_DAY) - - -def best_case_event_lrvs(best_case_lrv: float, treatments: list[Treatment], failing_events: list[np.ndarray | None], - group: PathogenGroup, shape: tuple) -> np.ndarray: - """Best-case LRV of each exposure event: on a failure day, the LRV of the mixed water under the mixed-water - assumption, from the steps that lose their positive maximum LRV for this pathogen group. Each combination of - failing steps is calculated once.""" - event_lrvs = np.full(shape, best_case_lrv, dtype=float) - steps = failing_steps(treatments, failing_events, group, "max") - if not steps: - return event_lrvs - # which of these steps fail on each event's day, as bits of one number - combination = np.zeros(shape, dtype=np.int64) - for bit, (failing, _, _) in enumerate(steps): - combination |= failing.astype(np.int64) << bit - for combination_code in np.unique(combination): - if combination_code == 0: - continue - failing_that_day = [(lost, duration) for bit, (_, lost, duration) in enumerate(steps) - if combination_code >> bit & 1] - event_lrvs[combination == combination_code] = best_case_lrv - mixed_water_lrv_loss(failing_that_day) + failing_share = failure_duration / MINUTES_PER_DAY + daily_lrv = -np.log10(failing_share + (1 - failing_share) * 10 ** -lrv) + return lrv - daily_lrv + + +def event_lrvs_of_case(train_lrv: float, treatments: list[Treatment], failing_events: list[np.ndarray | None], + group: PathogenGroup, bound: str, shape: tuple) -> np.ndarray: + """LRV of each exposure event in one case: best-case starts from every step's maximum LRV, worst-case from its + minimum LRV. On a failure day each failing step loses what its daily LRV lacks, so combined failures add up + without further rules (D6).""" + event_lrvs = np.full(shape, train_lrv, dtype=float) + for failing, lrv, failure_duration in failing_steps(treatments, failing_events, group, bound): + event_lrvs[failing] -= daily_lrv_loss(lrv, failure_duration) return event_lrvs @@ -176,10 +144,10 @@ def to_dalys(pr, pat=pathogen): log_concentrations, sample_positions = sample_exposure_events( inflow.min, inflow.max, risk_assessment.events_per_year) event_lrvs = { - BEST_CASE: best_case_event_lrvs(lrvs[group]["max"], treatments, failing_events, group, - sample_positions.shape), - WORST_CASE: worst_case_event_lrvs(lrvs[group]["min"], treatments, failing_events, group, - sample_positions.shape), + BEST_CASE: event_lrvs_of_case(lrvs[group]["max"], treatments, failing_events, group, "max", + sample_positions.shape), + WORST_CASE: event_lrvs_of_case(lrvs[group]["min"], treatments, failing_events, group, "min", + sample_positions.shape), } cases = {case: get_annual_risk(log_concentrations, sample_positions, lrvs_of_case, risk_assessment.volume_per_event, dist) diff --git a/qmra/risk_assessment/tests/test_assess_risk.py b/qmra/risk_assessment/tests/test_assess_risk.py index a4729b6..586a060 100644 --- a/qmra/risk_assessment/tests/test_assess_risk.py +++ b/qmra/risk_assessment/tests/test_assess_risk.py @@ -284,7 +284,7 @@ def case_statistics(result: dict, case: str) -> list[float]: WEAK_STEP = dict(bacteria_min=2, bacteria_max=3, viruses_min=2, viruses_max=3, protozoa_min=2, protozoa_max=3) -class TestFailureDaysWorstCase(TestCase): +class TestFailureDays(TestCase): def test_failure_frequency_0_gives_the_results_without_failure_inputs(self): without = calculate([STRONG_STEP, WEAK_STEP]) @@ -293,15 +293,6 @@ def test_failure_frequency_0_gives_the_results_without_failure_inputs(self): assert_that(with_zero).is_equal_to(without) - def test_always_failing_step_loses_its_positive_minimum_lrvs_in_worst_case(self): - failing = calculate([dict(STRONG_STEP, failure_frequency=365), WEAK_STEP]) - without_min_lrvs = calculate([dict(STRONG_STEP, bacteria_min=0, viruses_min=0, protozoa_min=0), WEAK_STEP]) - - for pathogen in PATHOGENS: - with self.subTest(pathogen): - assert_that(case_statistics(failing[pathogen], "minimum_lrv")).is_equal_to( - case_statistics(without_min_lrvs[pathogen], "minimum_lrv")) - def test_higher_failure_frequency_never_lowers_the_risk(self): results = [calculate([dict(STRONG_STEP, failure_frequency=f), WEAK_STEP]) for f in [0, 1, 10, 100, 365]] @@ -318,14 +309,6 @@ def test_higher_failure_frequency_never_lowers_the_risk(self): worst_median = [r["Rotavirus"]["infection_minimum_lrv_median"] for r in results] assert_that(worst_median[-1]).is_greater_than(worst_median[0]) - def test_failure_duration_does_not_change_worst_case(self): - short = calculate([dict(STRONG_STEP, failure_frequency=50, failure_duration=1), WEAK_STEP]) - long = calculate([dict(STRONG_STEP, failure_frequency=50, failure_duration=1440), WEAK_STEP]) - - for pathogen in PATHOGENS: - assert_that(case_statistics(long[pathogen], "minimum_lrv")).is_equal_to( - case_statistics(short[pathogen], "minimum_lrv")) - def test_worst_case_is_never_below_best_case(self): for frequencies in [(0, 0), (0, 5), (20, 5), (365, 5)]: results = calculate([dict(STRONG_STEP, failure_frequency=frequencies[0]), @@ -377,13 +360,7 @@ def assert_best_case_close(results: dict, reference: dict, pathogens: list[str]) BASE_STEP = all_groups(3, 4) -class TestMixedWaterBestCase(TestCase): - - def test_always_failing_full_day_loses_the_positive_maximum_lrvs_in_best_case(self): - failing = calculate([dict(STRONG_STEP, failure_frequency=365, failure_duration=1440), WEAK_STEP]) - without_max_lrvs = calculate([dict(STRONG_STEP, bacteria_max=0, viruses_max=0, protozoa_max=0), WEAK_STEP]) - - assert_best_case_close(failing, without_max_lrvs, PATHOGENS) +class TestMixedWaterEq5(TestCase): def test_always_failing_part_of_the_day_gives_the_mixed_lrv_of_eq_5(self): failing = calculate([dict(all_groups(0, 4), failure_frequency=365, failure_duration=60), BASE_STEP]) @@ -392,63 +369,84 @@ def test_always_failing_part_of_the_day_gives_the_mixed_lrv_of_eq_5(self): assert_best_case_close(failing, reference, PATHOGENS) - def test_two_steps_over_a_day_overlap_by_the_extra_minutes(self): - failing = calculate([dict(all_groups(0, 4), failure_frequency=365, failure_duration=900), - dict(all_groups(0, 3), failure_frequency=365, failure_duration=900), BASE_STEP]) - # 360 minutes both down, 540 minutes each alone - overlap_360 = mixed_lrv(11, [(360, 7), (540, 4), (540, 3)]) - reference = calculate([all_groups(0, overlap_360)]) - assert_best_case_close(failing, reference, PATHOGENS) +# the two steps of docs/failure-daily-lrv-example.md, with the same LRV in both cases +EXAMPLE_STEP_A = all_groups(4, 4) +EXAMPLE_STEP_B = all_groups(3, 3) +# never fails and keeps the risk below saturation, where a wrong train LRV would hardly change the results +NEVER_FAILING_LRV = 5 +# the worked example gives LRVs to two decimals: at most 0.005 log, a factor of 10 ** 0.005 on the concentration +ROUNDING = 10 ** 0.005 - 1 - def test_three_steps_within_a_day_do_not_overlap(self): - failing = calculate([dict(all_groups(0, 1), failure_frequency=365, failure_duration=400), - dict(all_groups(0, 1), failure_frequency=365, failure_duration=400), - dict(all_groups(0, 3), failure_frequency=365, failure_duration=400), BASE_STEP]) - no_overlap = mixed_lrv(9, [(400, 1), (400, 1), (400, 3)]) - reference = calculate([all_groups(0, no_overlap)]) - assert_best_case_close(failing, reference, PATHOGENS) +def assert_both_cases_close(results: dict, reference: dict, relative: float): + for pathogen in PATHOGENS: + for case in ["minimum_lrv", "maximum_lrv"]: + for value, expected in zip(case_statistics(results[pathogen], case), + case_statistics(reference[pathogen], case)): + assert_that(value).described_as(f"{pathogen}, {case}").is_close_to(expected, abs(expected) * relative) - def test_three_steps_over_a_day_count_like_worst_case(self): - failing = calculate([dict(all_groups(0, 1), failure_frequency=365, failure_duration=800), - dict(all_groups(0, 1), failure_frequency=365, failure_duration=800), - dict(all_groups(0, 3), failure_frequency=365, failure_duration=400), BASE_STEP]) - # all three steps down for the whole day: only the step that never fails is left - reference = calculate([BASE_STEP]) - assert_best_case_close(failing, reference, PATHOGENS) +class TestDailyLrv(TestCase): - def test_only_steps_that_lose_lrv_for_the_pathogen_group_count(self): - # the third step removes no protozoa, so for protozoa this is a day with two failing steps - steps = [all_groups(0, 1), all_groups(0, 1), dict(all_groups(0, 1), protozoa_max=0)] - failing = calculate([dict(step, failure_frequency=365, failure_duration=900) for step in steps] + [BASE_STEP]) - two_steps_overlapping = mixed_lrv(6, [(360, 2), (540, 1), (540, 1)]) - reference = calculate([dict(all_groups(3, 4), protozoa_min=0, protozoa_max=two_steps_overlapping)]) + def test_combined_failures_match_the_worked_example_in_both_cases(self): + for minutes, train_lrv_of_the_day in [(60, 2.75), (240, 1.55), (720, 0.60), (960, 0.35)]: + with self.subTest(minutes=minutes): + failing = calculate([dict(EXAMPLE_STEP_A, failure_frequency=365, failure_duration=minutes), + dict(EXAMPLE_STEP_B, failure_frequency=365, failure_duration=minutes), + all_groups(NEVER_FAILING_LRV, NEVER_FAILING_LRV)]) + train_lrv = NEVER_FAILING_LRV + train_lrv_of_the_day + reference = calculate([all_groups(train_lrv, train_lrv)]) - assert_best_case_close(failing, reference, PATHOGENS) + assert_both_cases_close(failing, reference, ROUNDING) - def test_three_steps_crossing_a_day_raise_best_case(self): - within, over = [calculate([dict(all_groups(0, 1), failure_frequency=100, failure_duration=duration)] * 3 + [BASE_STEP]) - for duration in [480, 481]] + def test_one_failing_step_matches_the_worked_example_in_both_cases(self): + failing = calculate([dict(EXAMPLE_STEP_A, failure_frequency=365, failure_duration=60), EXAMPLE_STEP_B]) + reference = calculate([all_groups(4.38, 4.38)]) - for pathogen in PATHOGENS: - for before, after in zip(case_statistics(within[pathogen], "maximum_lrv"), - case_statistics(over[pathogen], "maximum_lrv")): - assert_that(after).is_greater_than_or_equal_to(before) - assert_that(over["Rotavirus"]["infection_maximum_lrv_median"]).is_greater_than( - within["Rotavirus"]["infection_maximum_lrv_median"]) + assert_both_cases_close(failing, reference, ROUNDING) + + def test_always_failing_full_day_loses_the_positive_lrvs_in_both_cases(self): + failing = calculate([dict(STRONG_STEP, failure_frequency=365, failure_duration=1440), WEAK_STEP]) + without_the_step = calculate([WEAK_STEP]) - def test_longer_failure_duration_never_lowers_best_case(self): + assert_both_cases_close(failing, without_the_step, 1e-9) + + def test_longer_failure_duration_never_lowers_either_case(self): results = [calculate([dict(STRONG_STEP, failure_frequency=100, failure_duration=duration), dict(WEAK_STEP, failure_frequency=100, failure_duration=duration)]) for duration in [1, 60, 600, 1000, 1440]] for pathogen in PATHOGENS: - for pair, (shorter, longer) in enumerate(zip(results, results[1:])): - with self.subTest(pathogen=pathogen, pair=pair): - for short, long in zip(case_statistics(shorter[pathogen], "maximum_lrv"), - case_statistics(longer[pathogen], "maximum_lrv")): - assert_that(long).is_greater_than_or_equal_to(short) - best_median = [r["Rotavirus"]["infection_maximum_lrv_median"] for r in results] - assert_that(best_median[-1]).is_greater_than(best_median[0]) + for case in ["minimum_lrv", "maximum_lrv"]: + for pair, (shorter, longer) in enumerate(zip(results, results[1:])): + with self.subTest(pathogen=pathogen, case=case, pair=pair): + for short, long in zip(case_statistics(shorter[pathogen], case), + case_statistics(longer[pathogen], case)): + assert_that(long).is_greater_than_or_equal_to(short) + for case in ["minimum_lrv", "maximum_lrv"]: + median = [r["Rotavirus"][f"infection_{case}_median"] for r in results] + assert_that(median[-1]).described_as(case).is_greater_than(median[0]) + + def test_the_daily_lrv_is_averaged_by_concentration_not_by_lrv(self): + failing = calculate([dict(EXAMPLE_STEP_A, failure_frequency=365, failure_duration=60), + dict(EXAMPLE_STEP_B, failure_frequency=365, failure_duration=60)]) + # the plain average of the LRVs over the day, which the worked example rejects + lrv_average = calculate([all_groups(6.71, 6.71)]) + + for pathogen in PATHOGENS: + for case in ["minimum_lrv", "maximum_lrv"]: + assert_that(failing[pathogen][f"infection_{case}_median"]).described_as( + f"{pathogen}, {case}").is_greater_than(100 * lrv_average[pathogen][f"infection_{case}_median"]) + + def test_step_with_minimum_lrv_0_changes_only_best_case_when_it_fails(self): + step = all_groups(0, 4) + normal = calculate([step, WEAK_STEP]) + failing = calculate([dict(step, failure_frequency=365, failure_duration=600), WEAK_STEP]) + + for pathogen in PATHOGENS: + with self.subTest(pathogen): + assert_that(case_statistics(failing[pathogen], "minimum_lrv")).is_equal_to( + case_statistics(normal[pathogen], "minimum_lrv")) + assert_that(failing[pathogen]["infection_maximum_lrv_median"]).is_greater_than( + normal[pathogen]["infection_maximum_lrv_median"]) From ff9891dbfe9c0ea18e24692e7b201f274d44fc84 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:38:55 +0000 Subject: [PATCH 34/43] feat: the failure LRV follows from the number of parallel units (#39) Ticket 17 (D7, ADR-0002): a failing step with N parallel units keeps the share of water treated by the other units during the failure event. Its failure LRV, the concentration mix of untreated water (1/N) and water treated with the step's LRV, replaces LRV 0 in the daily LRV of both cases. One unit gives exactly the results of ticket 16. Tests compare with the D7 table for normal LRV 4 and 1, and check that more units never raise the risk. Calculation time locally, drinking water with 365 events: 0.069 s without failures, 0.117 s with failures in all five steps. Co-Authored-By: Claude Opus 5.5 --- .../17-c2-failure-lrv-from-parallel-units.md | 10 +++--- qmra/risk_assessment/risk.py | 27 ++++++++++------ .../risk_assessment/tests/test_assess_risk.py | 32 +++++++++++++++++++ 3 files changed, 55 insertions(+), 14 deletions(-) diff --git a/docs/tickets/17-c2-failure-lrv-from-parallel-units.md b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md index fad67fe..82c504a 100644 --- a/docs/tickets/17-c2-failure-lrv-from-parallel-units.md +++ b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md @@ -12,11 +12,11 @@ Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-fail **Status:** ready-for-agent -- [ ] The calculation reads each treatment step's number of parallel units and uses the failure LRV in the daily LRV of both cases. -- [ ] A step with one parallel unit gives the same results as in Ticket 16. -- [ ] A step with LRV 4 always failing for the whole day with 2, 4 or 10 parallel units gives the same results as a train in which that step has LRV 0.30, 0.60 or 1.00 (the D7 table, allowing for its two-decimal rounding). -- [ ] Raising the number of parallel units never raises any mean. -- [ ] All tests pass, and the export package keeps its files and columns with failures. +- [x] The calculation reads each treatment step's number of parallel units and uses the failure LRV in the daily LRV of both cases. +- [x] A step with one parallel unit gives the same results as in Ticket 16. +- [x] A step with LRV 4 always failing for the whole day with 2, 4 or 10 parallel units gives the same results as a train in which that step has LRV 0.30, 0.60 or 1.00 (the D7 table, allowing for its two-decimal rounding). +- [x] Raising the number of parallel units never raises any mean. +- [x] All tests pass, and the export package keeps its files and columns with failures. - [ ] The calculation time of a drinking-water scenario (365 events per year) with failures is measured and recorded in the C2 pull request. - [ ] Manual check on the dev environment, announced to the team first: - Enter the worked example's two steps with failures and run them as a guest and as a registered user. diff --git a/qmra/risk_assessment/risk.py b/qmra/risk_assessment/risk.py index 505f00f..2b5a258 100644 --- a/qmra/risk_assessment/risk.py +++ b/qmra/risk_assessment/risk.py @@ -51,25 +51,34 @@ def draw_failing_events(treatments: list[Treatment], events_per_year: int, def failing_steps(treatments: list[Treatment], failing_events: list[np.ndarray | None], group: PathogenGroup, - bound: str) -> list[tuple[np.ndarray, float, int]]: + bound: str) -> list[tuple[np.ndarray, float, int, int]]: """The steps that fail on some days and then lose LRV for this pathogen group, as (their failing events, their - minimum or maximum LRV, failure duration). A step whose LRV is 0 or below loses nothing when it fails (D4).""" + minimum or maximum LRV, failure duration, number of parallel units). A step whose LRV is 0 or below loses nothing + when it fails (D4).""" steps = [] for t, failing in zip(treatments, failing_events): lrv = step_lrv(t, group, bound) if failing is not None and lrv > 0: - steps.append((failing, lrv, t.failure_duration)) + steps.append((failing, lrv, t.failure_duration, t.parallel_units)) return steps -def daily_lrv_loss(lrv: float, failure_duration: int) -> float: +def failure_lrv(lrv: float, parallel_units: int) -> float: + """LRV of a step while one of its parallel units is down (D7, ADR-0002): that unit's share of the water is not + treated, the others' is treated with the step's LRV, mixed by concentration. 0 for one unit, at most log10 of the + number of units.""" + return -np.log10((parallel_units - 1) / parallel_units * 10 ** -lrv + 1 / parallel_units) + + +def daily_lrv_loss(lrv: float, failure_duration: int, parallel_units: int) -> float: """How much LRV a failing step loses on a failure day under the mixed-water assumption (ADR-0002). - The consumed water mixes water treated during the failure event, with LRV 0, and water treated in normal - operation, in proportion to the failure duration. The mixture is averaged by concentration, never by LRV. + The consumed water mixes water treated during the failure event, with the failure LRV, and water treated in + normal operation, in proportion to the failure duration. The mixture is averaged by concentration, never by LRV. """ failing_share = failure_duration / MINUTES_PER_DAY - daily_lrv = -np.log10(failing_share + (1 - failing_share) * 10 ** -lrv) + daily_lrv = -np.log10(failing_share * 10 ** -failure_lrv(lrv, parallel_units) + + (1 - failing_share) * 10 ** -lrv) return lrv - daily_lrv @@ -79,8 +88,8 @@ def event_lrvs_of_case(train_lrv: float, treatments: list[Treatment], failing_ev minimum LRV. On a failure day each failing step loses what its daily LRV lacks, so combined failures add up without further rules (D6).""" event_lrvs = np.full(shape, train_lrv, dtype=float) - for failing, lrv, failure_duration in failing_steps(treatments, failing_events, group, bound): - event_lrvs[failing] -= daily_lrv_loss(lrv, failure_duration) + for failing, lrv, failure_duration, parallel_units in failing_steps(treatments, failing_events, group, bound): + event_lrvs[failing] -= daily_lrv_loss(lrv, failure_duration, parallel_units) return event_lrvs diff --git a/qmra/risk_assessment/tests/test_assess_risk.py b/qmra/risk_assessment/tests/test_assess_risk.py index 586a060..8cd800a 100644 --- a/qmra/risk_assessment/tests/test_assess_risk.py +++ b/qmra/risk_assessment/tests/test_assess_risk.py @@ -450,3 +450,35 @@ def test_step_with_minimum_lrv_0_changes_only_best_case_when_it_fails(self): case_statistics(normal[pathogen], "minimum_lrv")) assert_that(failing[pathogen]["infection_maximum_lrv_median"]).is_greater_than( normal[pathogen]["infection_maximum_lrv_median"]) + + +class TestParallelUnits(TestCase): + + def test_failure_lrv_matches_the_d7_table(self): + # docs/review-2026-10-dev-scenarios.md, F2: the LRV a step keeps while one of its units is down + for normal_lrv, units, expected_lrv in [(4, 2, 0.30), (4, 4, 0.60), (4, 10, 1.00), + (1, 2, 0.26), (1, 4, 0.49), (1, 10, 0.72)]: + with self.subTest(normal_lrv=normal_lrv, units=units): + failing = calculate([dict(all_groups(normal_lrv, normal_lrv), failure_frequency=365, + failure_duration=1440, parallel_units=units), + all_groups(NEVER_FAILING_LRV, NEVER_FAILING_LRV)]) + train_lrv = NEVER_FAILING_LRV + expected_lrv + reference = calculate([all_groups(train_lrv, train_lrv)]) + + assert_both_cases_close(failing, reference, ROUNDING) + + def test_more_parallel_units_never_raise_the_risk(self): + results = [calculate([dict(STRONG_STEP, failure_frequency=100, failure_duration=600, parallel_units=units), + dict(WEAK_STEP, failure_frequency=100, failure_duration=600, parallel_units=units)]) + for units in [1, 2, 4, 10, 1000]] + + for pathogen in PATHOGENS: + for case in ["minimum_lrv", "maximum_lrv"]: + for pair, (fewer, more) in enumerate(zip(results, results[1:])): + with self.subTest(pathogen=pathogen, case=case, pair=pair): + for with_fewer, with_more in zip(case_statistics(fewer[pathogen], case), + case_statistics(more[pathogen], case)): + assert_that(with_more).is_less_than_or_equal_to(with_fewer) + for case in ["minimum_lrv", "maximum_lrv"]: + median = [r["Rotavirus"][f"infection_{case}_median"] for r in results] + assert_that(median[-1]).described_as(case).is_less_than(median[0]) From 4bb2649e1985677981428c4ba7e35210b4c80c78 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 10:54:32 +0000 Subject: [PATCH 35/43] docs: tick the manual check on dev of C2 ticket 17 (#39) [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/tickets/17-c2-failure-lrv-from-parallel-units.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tickets/17-c2-failure-lrv-from-parallel-units.md b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md index 82c504a..89535ca 100644 --- a/docs/tickets/17-c2-failure-lrv-from-parallel-units.md +++ b/docs/tickets/17-c2-failure-lrv-from-parallel-units.md @@ -18,7 +18,7 @@ Source: Spec C2 Failure calculation, revision for D6 and D7 (`docs/specs/c2-fail - [x] Raising the number of parallel units never raises any mean. - [x] All tests pass, and the export package keeps its files and columns with failures. - [ ] The calculation time of a drinking-water scenario (365 events per year) with failures is measured and recorded in the C2 pull request. -- [ ] Manual check on the dev environment, announced to the team first: +- [x] Manual check on the dev environment, announced to the team first (2026-10-17, passed): - Enter the worked example's two steps with failures and run them as a guest and as a registered user. - Change the failure duration and the number of parallel units, and see both cases react. - Confirm that the result page, the saved-assessments page and the export's result table show the changed results. From 464b31c83e18d7ae1d8e9533c85fea845bf3f8ea Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 11:03:18 +0000 Subject: [PATCH 36/43] Merge c2-failure-calculation into c3-failure-inputs-in-export: tickets 16-17, daily LRV and parallel units [skip ci] The reference export is updated on purpose. The benchmark assessment has a UV step failing 10 days per year for 60 minutes. Under D6 (ADR-0002) worst-case uses the failure duration instead of losing the step for the whole day, so only the worst-case ("Minimum LRV") rows of the result table change, all lower; the reference-level exceedance in the report drops from "min" to "none" twice. Best-case values, files, columns and layout are unchanged. Co-Authored-By: Claude Opus 5.5 --- .../package/Reference assessment-report.html | 4 ++-- .../package/Reference assessment-result.csv | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html index ab0890b..78052ee 100644 --- a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-report.html @@ -73,8 +73,8 @@

Result

Campylobacter jejuni - min - min + none + none Cryptosporidium parvum diff --git a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv index 36ceeb2..c014601 100644 --- a/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv +++ b/qmra/risk_assessment/tests/reference_export/package/Reference assessment-result.csv @@ -5,8 +5,8 @@ ,,Infection prob.,Infection prob.,Infection prob.,Infection prob.,Infection prob.,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy,DALYs pppy pathogen,stat,min,25%,50%,75%,max,min,25%,50%,75%,max Rotavirus,Maximum LRV,9.572485276665432e-07,1.4836628104142324e-06,1.7097025438728153e-06,1.931930578930041e-06,2.893425146477391e-06,6.700739693665803e-09,1.0385639672899628e-08,1.1967917807109707e-08,1.3523514052510289e-08,2.0253976025341736e-08 -Rotavirus,Minimum LRV,0.3640096391082689,0.41122481268750266,0.4267809590038193,0.44364273411319005,0.5027473445430916,0.002548067473757882,0.002878573688812519,0.002987466713026735,0.0031054991387923303,0.003519231411801641 +Rotavirus,Minimum LRV,0.3369085247433362,0.35979365904139426,0.36594592532905473,0.3723102321920796,0.40214153019360643,0.0023583596732033537,0.0025185556132897597,0.002561621477303383,0.0026061716253445574,0.002814990711355245 Campylobacter jejuni,Maximum LRV,4.5133408121955654e-10,2.2749717942804892e-09,3.0147686569392818e-09,3.744822868467779e-09,7.008461921920173e-09,6.22841032082988e-13,3.139461076107075e-12,4.160380746576208e-12,5.167855558485534e-12,9.671677452249837e-12 -Campylobacter jejuni,Minimum LRV,0.0001083267586116099,0.0005457054173893527,0.0007230268318788591,0.0008980393643011064,0.0016799226351181806,1.4949092688402165e-07,7.530734759973068e-07,9.977770279928255e-07,1.2392943227355267e-06,2.3182932364630894e-06 +Campylobacter jejuni,Minimum LRV,5.1759555220742826e-06,2.3390058868616448e-05,3.077959172620437e-05,3.8124305017595006e-05,7.071393981616936e-05,7.142818620462509e-09,3.2278281238690695e-08,4.247583658216203e-08,5.26115409242811e-08,9.758523694631371e-08 Cryptosporidium parvum,Maximum LRV,9.220223140538053e-09,1.429064969138416e-08,1.6467869567993176e-08,1.860837314793251e-08,2.7869510144107323e-08,9.681234297564956e-12,1.5005182175953368e-11,1.7291263046392836e-11,1.9538791805329135e-11,2.926298565131269e-11 -Cryptosporidium parvum,Minimum LRV,0.0028129655229737205,0.009066386874888782,0.011592701982251985,0.01415027039915931,0.025193602188789632,2.9536137991224066e-06,9.51970621863322e-06,1.2172337081364584e-05,1.4857783919117274e-05,2.645328229822911e-05 +Cryptosporidium parvum,Minimum LRV,0.00127144552620162,0.0015495414870806967,0.0016603494898462245,0.001775666665946557,0.002235528714494728,1.335017802511701e-06,1.6270185614347315e-06,1.743366964338536e-06,1.8644499992438848e-06,2.347305150219464e-06 From ac39fe4f6ab95b43acbdd408fe473dbb09b4eada Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 11:03:33 +0000 Subject: [PATCH 37/43] docs: C2 tickets 16-17 built and checked on dev; reference export updated for D6 [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/progress.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/progress.md b/docs/progress.md index 201f939..34ad33b 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -11,12 +11,12 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs | Change | Branch (based on) | Pull request | Tickets | State | |---|---|---|---|---| | C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`); #36, #37 (`14`, `15`) | D7 rework built (tickets 14–15), tested manually on dev | -| C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | Built; reopened for D6/D7, spec and tickets not revised yet | +| C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | D6/D7 rework built (tickets 16–17, #38, #39), tested manually on dev | | C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open; reopened for D7 (units column), ticket not written yet | | C4 Explain treatment failure | not created yet (will be based on C2) | — | #32, #33 (`12`, `13`) | Tickets ready | | C5 Failure benchmark | — | — | — | Waits for decisions D1–D3 | -**Dev** (https://dev.qmra.org) runs C1 (PR #28, commit `f19a791`) since 2026-10-16. C2 and C3 are not on dev until they are pushed again. +**Dev** (https://dev.qmra.org) runs C1 + C2 (PR #29, commit `ff9891d`) since 2026-10-17. C3 is not on dev until it is pushed again. **Review of 2026-09-28** (by a teammate): - **Statuses:** tickets 04–11 are set to "review". @@ -31,6 +31,11 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs - **Configurator (#36, `e547ca1`):** each treatment step has a number of parallel units (a whole number of 1 or more, no upper limit, default 1) next to failure frequency and failure duration, for guests and registered users; the shared rule rejects values below 1 and fractions with one message; existing assessments get 1. - **Personal treatment steps (#37, `f19a791`):** they store the number of parallel units under the same rule, and the configurator copies it into a newly added step; existing personal treatment steps get 1. - **Manual check on dev** passed on 2026-10-17. C1 was merged forward into C2 and C3; their tests pass. +- **C2:** spec revised for D6 and D7 (`3196dc4`), tickets 16 (#38) and 17 (#39) written and built; tickets 08 and 09 are marked as partly replaced: + - **Daily LRV (#38, `99ad33d`):** on a failure day both cases use each failing step's daily LRV under the mixed-water assumption, averaged by concentration and starting from the maximum or minimum LRV; the losses of the failing steps are summed. The worst-case full-day loss and the overlap rules are removed. Tests compare with the worked example (`docs/failure-daily-lrv-example.md`). + - **Parallel units (#39, `ff9891d`):** the failure LRV follows from the number of parallel units; one unit gives LRV 0. Tests compare with the D7 table. + - **Calculation time** (local, drinking water, 365 events, 5 steps): 0.069 s without failures, 0.117 s with failures in all five steps. Still to be recorded in PR #29. + - **Manual check on dev** passed on 2026-10-17. C2 was merged forward into C3 (`464b31c`); the reference export was updated on purpose, see below. ## What each Change delivers @@ -68,6 +73,8 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs - `UPDATE_REFERENCE_EXPORT=1` rewrites the reference on purpose (see the README there). - **Failure columns (#31, `a04e2f9`):** `treatments.csv` has "Failure frequency (days per year)" and "Failure duration (minutes)", repeated on each step's three rows. The HTML report stays without inputs. - **For the release notes:** `treatments.csv` has two new columns at the end. +- **Reference export updated on 2026-10-17 (`464b31c`):** after D6, the worst-case ("Minimum LRV") results of the benchmark assessment are lower, because worst-case now uses the failure duration; the reference-level exceedance in the report drops from "min" to "none" twice. Best-case values, files and columns are unchanged. +- **For the release notes:** worst-case results of assessments with failures are lower than with the first version (D6, ADR-0002); the reference-level exceedance can drop from "min" to "none". ## Decisions made during implementation From 98706d0dc4cc8cffa8e030f033aa120f1b8e3181 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 11:09:34 +0000 Subject: [PATCH 38/43] docs: add C3 ticket 18, number of parallel units in the export [skip ci] Co-Authored-By: Claude Opus 5.5 --- .../tickets/18-c3-parallel-units-in-export.md | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 docs/tickets/18-c3-parallel-units-in-export.md diff --git a/docs/tickets/18-c3-parallel-units-in-export.md b/docs/tickets/18-c3-parallel-units-in-export.md new file mode 100644 index 0000000..55582eb --- /dev/null +++ b/docs/tickets/18-c3-parallel-units-in-export.md @@ -0,0 +1,28 @@ +# 18: Number of parallel units in the treatment table of the export + +**What to build:** The export package records the third failure input as well (D7). The treatment table `treatments.csv` gets the column "Number of parallel units" at the end, after "Failure frequency (days per year)" and "Failure duration (minutes)". Like the other two failure inputs, it is repeated on each of the step's three rows (one per pathogen group), so the file keeps its layout of one row per step and pathogen group. + +The HTML report stays as it is: it shows no inputs at all, so it doesn't show failure inputs either (decided 2026-09-25). + +This is an intended change to the export package, so the reference export is updated in the same commit, by the new column only. The benchmark assessment's inputs stay unchanged, so its results don't change and its step shows 1 parallel unit. The change is noted for the release notes in `docs/progress.md` and in the C3 pull request. + +This Ticket also completes the manual check on dev that Ticket 11 left open. + +Delivered on branch `c3-failure-inputs-in-export` as part of the C3 pull request (#34). The pull request is tested on the dev environment again and not merged before the release (D5). + +Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), roadmap decision D7, decisions of 2026-09-25 (report without inputs; failure columns repeated on the step's rows). GitHub issue: not created yet. + +**Blocked by:** None (can start immediately; Tickets 14 and 15, which store the number of parallel units, are merged into this branch) + +**Status:** ready-for-agent + +- [ ] `treatments.csv` has the column "Number of parallel units" after the two failure columns, with each step's value on all three of its rows. +- [ ] An assessment saved with a number of parallel units exports exactly those values per step, and an assessment saved before C1 exports 1. +- [ ] No other file of the export package changes; the HTML report shows no inputs, as before. +- [ ] The reference export is updated in the same commit, by the new column only, and the reference test passes. +- [ ] The change is noted for the release notes in `docs/progress.md` and in the C3 pull request. +- [ ] Manual check on the dev environment, announced to the team first (also completes Ticket 11's open check): + - Download the export of an assessment with failure inputs, including a number of parallel units above 1. + - Check the three failure columns in `treatments.csv` and that the report is unchanged. + +Quality goals (`docs/vision.md`): Traceability (a reader can see which failure inputs a result is based on), Export stability (the reference export changes only on purpose, by the new column). From 9e2f057177ecfbb71206b15273f51bc83acc2653 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 11:14:09 +0000 Subject: [PATCH 39/43] docs: link C3 ticket 18 to issue #40 [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/review-2026-10-dev-scenarios.md | 1 + docs/tickets/18-c3-parallel-units-in-export.md | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/review-2026-10-dev-scenarios.md b/docs/review-2026-10-dev-scenarios.md index a6a6950..954662e 100644 --- a/docs/review-2026-10-dev-scenarios.md +++ b/docs/review-2026-10-dev-scenarios.md @@ -139,3 +139,4 @@ Answers (2026-10-16): | 15 (#37) Personal treatment steps store the number of parallel units | F2 (D7) | C1 | `c1-failure-inputs` | | 16 (#38) Both cases use the daily LRV of each failing step | F1 (D6) | C2 | `c2-failure-calculation` | | 17 (#39) The failure LRV follows from the number of parallel units | F2 (D7) | C2 | `c2-failure-calculation` | +| 18 (#40) Number of parallel units in the treatment table of the export | F2 (D7) | C3 | `c3-failure-inputs-in-export` | diff --git a/docs/tickets/18-c3-parallel-units-in-export.md b/docs/tickets/18-c3-parallel-units-in-export.md index 55582eb..51ad2bb 100644 --- a/docs/tickets/18-c3-parallel-units-in-export.md +++ b/docs/tickets/18-c3-parallel-units-in-export.md @@ -10,7 +10,7 @@ This Ticket also completes the manual check on dev that Ticket 11 left open. Delivered on branch `c3-failure-inputs-in-export` as part of the C3 pull request (#34). The pull request is tested on the dev environment again and not merged before the release (D5). -Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), roadmap decision D7, decisions of 2026-09-25 (report without inputs; failure columns repeated on the step's rows). GitHub issue: not created yet. +Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), roadmap decision D7, decisions of 2026-09-25 (report without inputs; failure columns repeated on the step's rows). GitHub issue #40. **Blocked by:** None (can start immediately; Tickets 14 and 15, which store the number of parallel units, are merged into this branch) From dfa309fb5d1a4f767cba538a5816f9c0b181eba3 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 11:19:57 +0000 Subject: [PATCH 40/43] feat: number of parallel units in the treatment table of the export (#40) Ticket 18 (D7): treatments.csv gets the column "Number of parallel units" after the two failure columns, repeated on each step's three rows. Assessments saved before C1 export 1. The HTML report stays without inputs. The reference export is updated on purpose in the same commit: only treatments.csv changes, by the new header and ",1" on every row; the benchmark assessment's inputs and results are unchanged. The release note in docs/progress.md now names three new columns. Co-Authored-By: Claude Opus 5.5 --- docs/progress.md | 2 +- .../tickets/18-c3-parallel-units-in-export.md | 8 ++--- qmra/risk_assessment/exports.py | 1 + .../exposure-assessment/treatments.csv | 14 ++++----- .../tests/test_failure_inputs.py | 31 ++++++++++++++----- 5 files changed, 36 insertions(+), 20 deletions(-) diff --git a/docs/progress.md b/docs/progress.md index 34ad33b..36c7612 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -72,7 +72,7 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs - A fixed benchmark assessment is exported and compared with `qmra/risk_assessment/tests/reference_export/`: the same files, identical text, and each number within 0.1 % relative. Plots are only checked for presence. - `UPDATE_REFERENCE_EXPORT=1` rewrites the reference on purpose (see the README there). - **Failure columns (#31, `a04e2f9`):** `treatments.csv` has "Failure frequency (days per year)" and "Failure duration (minutes)", repeated on each step's three rows. The HTML report stays without inputs. -- **For the release notes:** `treatments.csv` has two new columns at the end. +- **For the release notes:** `treatments.csv` has three new columns at the end: "Failure frequency (days per year)", "Failure duration (minutes)" and, since ticket 18 (#40), "Number of parallel units". - **Reference export updated on 2026-10-17 (`464b31c`):** after D6, the worst-case ("Minimum LRV") results of the benchmark assessment are lower, because worst-case now uses the failure duration; the reference-level exceedance in the report drops from "min" to "none" twice. Best-case values, files and columns are unchanged. - **For the release notes:** worst-case results of assessments with failures are lower than with the first version (D6, ADR-0002); the reference-level exceedance can drop from "min" to "none". diff --git a/docs/tickets/18-c3-parallel-units-in-export.md b/docs/tickets/18-c3-parallel-units-in-export.md index 51ad2bb..eda3618 100644 --- a/docs/tickets/18-c3-parallel-units-in-export.md +++ b/docs/tickets/18-c3-parallel-units-in-export.md @@ -16,10 +16,10 @@ Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), roadmap decisio **Status:** ready-for-agent -- [ ] `treatments.csv` has the column "Number of parallel units" after the two failure columns, with each step's value on all three of its rows. -- [ ] An assessment saved with a number of parallel units exports exactly those values per step, and an assessment saved before C1 exports 1. -- [ ] No other file of the export package changes; the HTML report shows no inputs, as before. -- [ ] The reference export is updated in the same commit, by the new column only, and the reference test passes. +- [x] `treatments.csv` has the column "Number of parallel units" after the two failure columns, with each step's value on all three of its rows. +- [x] An assessment saved with a number of parallel units exports exactly those values per step, and an assessment saved before C1 exports 1. +- [x] No other file of the export package changes; the HTML report shows no inputs, as before. +- [x] The reference export is updated in the same commit, by the new column only, and the reference test passes. - [ ] The change is noted for the release notes in `docs/progress.md` and in the C3 pull request. - [ ] Manual check on the dev environment, announced to the team first (also completes Ticket 11's open check): - Download the export of an assessment with failure inputs, including a number of parallel units above 1. diff --git a/qmra/risk_assessment/exports.py b/qmra/risk_assessment/exports.py index f240e9c..7c68e74 100644 --- a/qmra/risk_assessment/exports.py +++ b/qmra/risk_assessment/exports.py @@ -31,6 +31,7 @@ def treatments_as_df(treatments: QuerySet[Treatment]) -> pd.DataFrame: # a step's failure inputs apply to all its pathogen groups "Failure frequency (days per year)": [t.failure_frequency] * 3, "Failure duration (minutes)": [t.failure_duration] * 3, + "Number of parallel units": [t.parallel_units] * 3, })] return pd.concat(dfs) diff --git a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv index f77a540..682355c 100644 --- a/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv +++ b/qmra/risk_assessment/tests/reference_export/package/exposure-assessment/treatments.csv @@ -1,7 +1,7 @@ -Treatment,Pathogen group,Maximum LRV,Minimum LRV,Failure frequency (days per year),Failure duration (minutes) -Slow sand filtration,Viruses,4.0,0.25,0.0,30 -Slow sand filtration,Bacteria,6.0,2.0,0.0,30 -Slow sand filtration,Protozoa,5.0,0.3,0.0,30 -"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0,10.0,60 -"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6,10.0,60 -"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5,10.0,60 +Treatment,Pathogen group,Maximum LRV,Minimum LRV,Failure frequency (days per year),Failure duration (minutes),Number of parallel units +Slow sand filtration,Viruses,4.0,0.25,0.0,30,1 +Slow sand filtration,Bacteria,6.0,2.0,0.0,30,1 +Slow sand filtration,Protozoa,5.0,0.3,0.0,30,1 +"UV disinfection 20 mJ/cm2, drinking",Viruses,3.0,1.0,10.0,60,1 +"UV disinfection 20 mJ/cm2, drinking",Bacteria,6.0,4.6,10.0,60,1 +"UV disinfection 20 mJ/cm2, drinking",Protozoa,3.0,2.5,10.0,60,1 diff --git a/qmra/risk_assessment/tests/test_failure_inputs.py b/qmra/risk_assessment/tests/test_failure_inputs.py index b00bd14..67128d3 100644 --- a/qmra/risk_assessment/tests/test_failure_inputs.py +++ b/qmra/risk_assessment/tests/test_failure_inputs.py @@ -263,11 +263,16 @@ def test_any_whole_number_of_parallel_units_of_1_or_more_is_accepted(self): assert_that(response.status_code).is_equal_to(302) - def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): + def assessment_from_before_failure_inputs(self) -> RiskAssessment: + """An assessment stored without failure inputs, as before C1: its treatment step has the defaults.""" assessment = RiskAssessment.objects.create(user=self.user, events_per_year=365, volume_per_event=1) for pathogen in ["Rotavirus", "Campylobacter jejuni", "Cryptosporidium parvum"]: Inflow.objects.create(risk_assessment=assessment, pathogen=pathogen, min=10, max=100) Treatment.objects.create(risk_assessment=assessment, name="Primary treatment", **LRVS) + return assessment + + def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): + assessment = self.assessment_from_before_failure_inputs() assert_that(failure_inputs_per_step(self.reopen(assessment))).is_equal_to([ dict(name="Primary treatment", failure_frequency=0, failure_duration=30, parallel_units=1) @@ -275,10 +280,12 @@ def test_assessment_from_before_failure_inputs_reopens_with_defaults(self): @staticmethod def exported_failure_inputs(export: dict) -> list[tuple]: - """(treatment, pathogen group, failure frequency, failure duration) of each row of the exported treatment table.""" + """(treatment, pathogen group, failure frequency, failure duration, number of parallel units) of each row of + the exported treatment table.""" table = pd.read_csv(io.BytesIO(export[TREATMENTS_CSV])) - return list(table[["Treatment", "Pathogen group", "Failure frequency (days per year)", - "Failure duration (minutes)"]].itertuples(index=False, name=None)) + columns = ["Treatment", "Pathogen group", "Failure frequency (days per year)", "Failure duration (minutes)", + "Number of parallel units"] + return list(table[columns].itertuples(index=False, name=None)) def saved_assessment_with_two_steps(self) -> RiskAssessment: data = new_configurator(self.client) @@ -300,7 +307,7 @@ def test_failure_frequency_0_changes_neither_results_nor_export_beyond_the_durat assert_that(self.results(assessment)).is_equal_to(results_before) # the export records the new durations in the treatment table; everything else stays identical export_after = self.export(assessment) - durations = {duration for *_, duration in self.exported_failure_inputs(export_after)} + durations = set(pd.read_csv(io.BytesIO(export_after[TREATMENTS_CSV]))["Failure duration (minutes)"]) export_after.pop(TREATMENTS_CSV) export_before.pop(TREATMENTS_CSV) assert_that(export_after).is_equal_to(export_before) @@ -308,14 +315,22 @@ def test_failure_frequency_0_changes_neither_results_nor_export_beyond_the_durat def test_export_records_the_failure_inputs_of_each_step(self): data = new_configurator(self.client) - add_step(data, "Primary treatment", **HIGH_LRVS, failure_frequency="12.5", failure_duration="90") + add_step(data, "Primary treatment", **HIGH_LRVS, failure_frequency="12.5", failure_duration="90", + parallel_units="3") add_step(data, "Slow sand filtration", **HIGH_LRVS) assessment = self.save_new(data) groups = ["Viruses", "Bacteria", "Protozoa"] assert_that(self.exported_failure_inputs(self.export(assessment))).is_equal_to( - [("Primary treatment", group, 12.5, 90) for group in groups] - + [("Slow sand filtration", group, 0, 30) for group in groups]) + [("Primary treatment", group, 12.5, 90, 3) for group in groups] + + [("Slow sand filtration", group, 0, 30, 1) for group in groups]) + + def test_export_of_an_assessment_from_before_failure_inputs_records_the_defaults(self): + assessment = self.assessment_from_before_failure_inputs() + + groups = ["Viruses", "Bacteria", "Protozoa"] + assert_that(self.exported_failure_inputs(self.export(assessment))).is_equal_to( + [("Primary treatment", group, 0, 30, 1) for group in groups]) def test_export_keeps_its_files_and_columns_with_failures(self): assessment = self.saved_assessment_with_two_steps() From 0ccc438f589bd5e47bd3a3e6cf0d5628d3566371 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 12:19:18 +0000 Subject: [PATCH 41/43] docs: C3 ticket 18 checked on dev, completing ticket 11's check [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/progress.md | 8 ++++++-- docs/tickets/11-c3-failure-inputs-in-export.md | 2 +- docs/tickets/18-c3-parallel-units-in-export.md | 2 +- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/progress.md b/docs/progress.md index 36c7612..d2a83fa 100644 --- a/docs/progress.md +++ b/docs/progress.md @@ -12,11 +12,11 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs |---|---|---|---|---| | C1 Failure inputs | `c1-failure-inputs` (`main`) | #28 | #21, #22, #23 (`docs/tickets/04`–`06`); #36, #37 (`14`, `15`) | D7 rework built (tickets 14–15), tested manually on dev | | C2 Failure calculation | `c2-failure-calculation` (C1) | #29 | #24, #25, #26 (`07`–`09`) | D6/D7 rework built (tickets 16–17, #38, #39), tested manually on dev | -| C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`) | Built, deployed to dev, manual check open; reopened for D7 (units column), ticket not written yet | +| C3 Failure inputs in the export | `c3-failure-inputs-in-export` (C2) | #34 | #30, #31 (`10`, `11`); #40 (`18`) | D7 rework built (ticket 18), tested manually on dev | | C4 Explain treatment failure | not created yet (will be based on C2) | — | #32, #33 (`12`, `13`) | Tickets ready | | C5 Failure benchmark | — | — | — | Waits for decisions D1–D3 | -**Dev** (https://dev.qmra.org) runs C1 + C2 (PR #29, commit `ff9891d`) since 2026-10-17. C3 is not on dev until it is pushed again. +**Dev** (https://dev.qmra.org) runs C1 + C2 + C3 (PR #34, commit `dfa309f`) since 2026-10-17. **Review of 2026-09-28** (by a teammate): - **Statuses:** tickets 04–11 are set to "review". @@ -36,6 +36,10 @@ targets `main` only so that the pipeline deploys it to dev. The production jobs - **Parallel units (#39, `ff9891d`):** the failure LRV follows from the number of parallel units; one unit gives LRV 0. Tests compare with the D7 table. - **Calculation time** (local, drinking water, 365 events, 5 steps): 0.069 s without failures, 0.117 s with failures in all five steps. Still to be recorded in PR #29. - **Manual check on dev** passed on 2026-10-17. C2 was merged forward into C3 (`464b31c`); the reference export was updated on purpose, see below. +- **C3:** ticket 18 (#40) written and built: + - **Parallel units column (#40, `dfa309f`):** `treatments.csv` has the column "Number of parallel units" after the two failure columns, repeated on each step's three rows; assessments saved before C1 export 1. The reference export was updated on purpose by that column only. + - **Manual check on dev** passed on 2026-10-17, which also completes Ticket 11's check of the two failure columns and the unchanged report. +- **Still open:** the release note in PR #34 and the calculation time in PR #29 (to be posted on GitHub); then C4 and C5. ## What each Change delivers diff --git a/docs/tickets/11-c3-failure-inputs-in-export.md b/docs/tickets/11-c3-failure-inputs-in-export.md index cca3996..8a5fb5a 100644 --- a/docs/tickets/11-c3-failure-inputs-in-export.md +++ b/docs/tickets/11-c3-failure-inputs-in-export.md @@ -23,4 +23,4 @@ Review on 2026-09-28 (commit `a04e2f9`): - [x] Only `treatments.csv` changed in the reference export, in the same commit as the code; later commits touched only the test tolerance and the README. - [x] Assessments saved before C1 export 0 and 30, because the new fields have those defaults. -- [ ] Manual check on dev from PR #34 still open: download an export with failure inputs, check the two columns and the unchanged report. +- [x] Manual check on dev from PR #34 (done 2026-10-17 together with Ticket 18, #40, passed): download an export with failure inputs, check the two columns and the unchanged report. diff --git a/docs/tickets/18-c3-parallel-units-in-export.md b/docs/tickets/18-c3-parallel-units-in-export.md index eda3618..c2bf781 100644 --- a/docs/tickets/18-c3-parallel-units-in-export.md +++ b/docs/tickets/18-c3-parallel-units-in-export.md @@ -21,7 +21,7 @@ Source: Change C3 in `docs/failure_roadmap.md` (Small, no Spec), roadmap decisio - [x] No other file of the export package changes; the HTML report shows no inputs, as before. - [x] The reference export is updated in the same commit, by the new column only, and the reference test passes. - [ ] The change is noted for the release notes in `docs/progress.md` and in the C3 pull request. -- [ ] Manual check on the dev environment, announced to the team first (also completes Ticket 11's open check): +- [x] Manual check on the dev environment, announced to the team first (also completes Ticket 11's open check) (2026-10-17, passed): - Download the export of an assessment with failure inputs, including a number of parallel units above 1. - Check the three failure columns in `treatments.csv` and that the report is unchanged. From d80a1f1c63a73f17b460dfaf4829333910ce1879 Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 12:27:30 +0000 Subject: [PATCH 42/43] docs: revise C4 tickets 12 and 13 for D6 and D7 [skip ci] Co-Authored-By: Claude Opus 5.5 --- .../12-c4-explain-failure-in-faq-and-tour.md | 24 +++++++++++-------- .../13-c4-sphinx-page-treatment-failure.md | 17 ++++++++----- 2 files changed, 25 insertions(+), 16 deletions(-) diff --git a/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md b/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md index 43bf0b4..47775c3 100644 --- a/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md +++ b/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md @@ -1,30 +1,34 @@ # 12: Explain treatment failure in the FAQ and the guided tour +**Revised 2026-10-17** for D6 and D7 (ADR-0002, `docs/review-2026-10-dev-scenarios.md`): the first version explained why worst-case ignores the failure duration and the overlap rules for combined failures; both were replaced. Not built yet. + **What to build:** Users of the application learn what the failure inputs mean and how the result uses them, without reading the method documents. - **FAQ:** a new question on treatment failure explains: - - failure frequency (failure days per year) and failure duration (minutes); + - the three failure inputs of a treatment step: failure frequency (failure days per year), failure duration (minutes) and number of parallel units; - that failure days are drawn at random per exposure event (ADR-0001); - - that a failure removes only the positive LRVs of a step (D4); - - why worst-case ignores the failure duration; - - the mixed-water assumption in best-case; - - combined failures: no overlap within a day, two steps overlapping by the minutes beyond a day, and three or more steps longer than a day in total counting like worst-case, counting only steps that lose LRV for the pathogen group. + - the failure LRV: a failure event stops one parallel unit and the others keep working, so a step with one unit loses its LRV during the failure, and a step with N units keeps at most log₁₀ N (for example 0.3 for two units) (D7); + - the daily LRV under the mixed-water assumption: the water used on a failure day is a mixture of water treated during the failure and in normal operation, in proportion to the failure duration; + - that best-case and worst-case use the same rule and differ only in the LRV a step starts from (its maximum or minimum LRV), so the failure duration affects both cases (D6); + - combined failures: the failing steps' daily LRVs add up, as if each failure happened at a random time of day; + - that a failure removes only the positive LRVs of a step (D4). - The limitations question gets one line: partial loss of removal isn't modelled. -- **Guided tour:** step 4 (Treatment) gets one or two sentences on the two failure fields. + The limitations question gets one line: units of a step that fail together, for example through a shared power supply, are not modelled; such a step is entered with one unit. +- **Guided tour:** step 4 (Treatment) gets one or two sentences on the three failure fields. The text uses the glossary terms from `CONTEXT.md`. It is a draft written by Claude and must be reviewed by a domain expert before the release. The pull request says so. Delivered on a new branch based on `c2-failure-calculation`, together with #33 as the C4 pull request (a draft into `main` only so that it deploys to dev; not merged before the release, D5). -Source: Change C4 in `docs/failure_roadmap.md` (Small, no Spec), `CONTEXT.md` (Treatment failure, Failure day, Failure frequency, Failure duration, Mixed-water assumption, Combined failure), ADR-0001, decisions 1–8 in `docs/QMRA_Failure_Approach.md`, the C2 Spec, and the decisions of 2026-09-25. GitHub issue #32. +Source: Change C4 in `docs/failure_roadmap.md` (Small, no Spec), `CONTEXT.md` (Treatment failure, Failure day, Failure frequency, Failure duration, Parallel units, Failure LRV, Mixed-water assumption, Daily LRV, Combined failure), ADR-0001, ADR-0002, roadmap decisions D4, D6 and D7, the worked example `docs/failure-daily-lrv-example.md`, and the C2 Spec. GitHub issue #32. **Blocked by:** None (can start immediately) **Status:** ready-for-agent - [ ] The FAQ has a question on treatment failure covering all points above, consistent with `CONTEXT.md` and the implemented calculation. -- [ ] The FAQ's limitations mention that partial loss of removal isn't modelled. -- [ ] Step 4 of the configurator's guided tour mentions failure frequency and failure duration with their units. +- [ ] The FAQ's limitations mention that units of a step failing together are not modelled and that such a step is entered with one unit. +- [ ] Step 4 of the configurator's guided tour mentions failure frequency, failure duration and the number of parallel units, with their units. +- [ ] Nothing in the FAQ or the tour says that worst-case ignores the failure duration or describes overlap rules for combined failures. - [ ] Deferred UI wording (such as "tolerable risk level") and explanations of unrelated parts of the model are left as they are. - [ ] The pull request marks the text as a draft for domain expert review. diff --git a/docs/tickets/13-c4-sphinx-page-treatment-failure.md b/docs/tickets/13-c4-sphinx-page-treatment-failure.md index 89696fb..c646602 100644 --- a/docs/tickets/13-c4-sphinx-page-treatment-failure.md +++ b/docs/tickets/13-c4-sphinx-page-treatment-failure.md @@ -1,14 +1,17 @@ # 13: Sphinx page Treatment failure +**Revised 2026-10-17** for D6 and D7 (ADR-0002, `docs/review-2026-10-dev-scenarios.md`): the first version described the worst-case full loss for the whole day, Eq. 5 in best-case only and the overlap rules for combined failures; these were replaced. Not built yet. + **What to build:** The Sphinx documentation (`docs/source/`) gets a page "Treatment failure", linked from `index.rst`. It explains the method in more detail than the FAQ: -- the inputs and their ranges; +- the three failure inputs and their ranges: failure frequency (0–365 days per year, default 0), failure duration (1–1,440 minutes, default 30) and number of parallel units (a whole number of 1 or more, default 1); - failure days per exposure event, and why the application departs from the 365-day sequence of the source method (ADR-0001); -- worst-case full loss for the whole day; -- the mixed-water assumption with Eq. 5; -- the rules for combined failures; -- D4 (only positive LRVs are lost). +- the failure LRV of a step with N parallel units, −log₁₀((N − 1)/N · 10^(−LRV) + 1/N), with the table of examples from the review (D7); +- the daily LRV under the mixed-water assumption, −log₁₀(d · 10^(−failure LRV) + (1 − d) · 10^(−LRV)) with d = failure duration / 1,440, averaged by concentration and never by LRV, used by both cases from the maximum or minimum LRV (D6); +- combined failures: the failing steps' daily LRVs add up, which equals placing each failure event at a random time of day, with the worked example's numbers; +- D4: only positive LRVs are affected; +- how the method differs from the source (Hambsch et al., 2019): per-event draws instead of 365 days (ADR-0001), and worst-case using the failure duration (ADR-0002). -It cites ADR-0001 and `docs/QMRA_Failure_Approach.md`. +It cites ADR-0001, ADR-0002, `docs/QMRA_Failure_Approach.md` and `docs/failure-daily-lrv-example.md`. The text is a draft written by Claude for domain expert review before the release. @@ -19,6 +22,8 @@ Source: Change C4 in `docs/failure_roadmap.md` (Small, no Spec), the same method **Status:** ready-for-agent - [ ] `docs/source/` has a page "Treatment failure" covering the points above, linked from `index.rst`. +- [ ] The formulas and the worked example's numbers on the page agree with ADR-0002, the worked example and the implemented calculation. +- [ ] The page describes no worst-case full loss regardless of the duration and no overlap rules for combined failures. - [ ] The documentation builds without errors, checked in a local Sphinx environment (Sphinx isn't in the requirements). - [ ] The page uses the glossary terms and agrees with `CONTEXT.md` and the implemented calculation. - [ ] The pull request marks the text as a draft for domain expert review. From ed2887968b6b0067ae7893c4f283032e8758645f Mon Sep 17 00:00:00 2001 From: ma-z-am Date: Sat, 17 Oct 2026 14:22:22 +0000 Subject: [PATCH 43/43] docs: base C4 on c3-failure-inputs-in-export [skip ci] Co-Authored-By: Claude Opus 5.5 --- docs/tickets/12-c4-explain-failure-in-faq-and-tour.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md b/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md index 47775c3..838ae71 100644 --- a/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md +++ b/docs/tickets/12-c4-explain-failure-in-faq-and-tour.md @@ -18,7 +18,7 @@ The text uses the glossary terms from `CONTEXT.md`. It is a draft written by Claude and must be reviewed by a domain expert before the release. The pull request says so. -Delivered on a new branch based on `c2-failure-calculation`, together with #33 as the C4 pull request (a draft into `main` only so that it deploys to dev; not merged before the release, D5). +Delivered on a new branch `c4-failure-explanations` based on `c3-failure-inputs-in-export` (so the C4 tickets and the export are there), together with #33 as the C4 pull request (a draft into `main` only so that it deploys to dev; not merged before the release, D5). Source: Change C4 in `docs/failure_roadmap.md` (Small, no Spec), `CONTEXT.md` (Treatment failure, Failure day, Failure frequency, Failure duration, Parallel units, Failure LRV, Mixed-water assumption, Daily LRV, Combined failure), ADR-0001, ADR-0002, roadmap decisions D4, D6 and D7, the worked example `docs/failure-daily-lrv-example.md`, and the C2 Spec. GitHub issue #32.