From 16097e784513df5840814aefed5d76563d11d34a Mon Sep 17 00:00:00 2001 From: Dave Wilding Date: Mon, 7 Sep 2026 10:01:26 +0800 Subject: [PATCH] Add guidance: choose observables that survive the full event sequence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When a test observes a side effect of an event handler (e.g. a status message), it must account for events that fire afterward. In charm frameworks, one event often triggers others — a config change can re-fire pebble-ready, which may overwrite the observable the test is waiting for. This caused a false negative in PR #73: the test waited for a status message set by config-changed, but pebble-ready fired afterward and cleared it. Added a generic principle to the test strategy section: trace the full event sequence and choose an observable that no later handler clobbers. --- .github/scripts/probe_issue.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.github/scripts/probe_issue.py b/.github/scripts/probe_issue.py index 8913bf1..3d9e9ac 100644 --- a/.github/scripts/probe_issue.py +++ b/.github/scripts/probe_issue.py @@ -408,6 +408,19 @@ def test_deploy(charm, juju: jubilant.Juju): kepler + kosmos; m- charms: meteor + micron) so the only meaningful \ difference is the configuration you changed. +**Choose observables that survive the full event sequence.** When your \ +test observes a side effect of an event handler (e.g. a status message, a \ +log record, a stored value), trace what happens *after* the event you are \ +testing. In charm frameworks, one event often triggers others — a config \ +change can re-fire `pebble-ready`, a relation change can trigger \ +`config-changed`, and so on. If a later handler overwrites or clears your \ +observable, your test will fail for reasons unrelated to the claim. Before \ +finalizing your test, read every handler in the charm and ask: "will any \ +other handler fire after the one I'm testing, and will it clobber what I'm \ +observing?" If so, choose a different observable (e.g. `StoredState`, a \ +file in the container, `workload_version`) or adjust the handler so it \ +preserves the observable. + ### Differential testing with xfail Sometimes a claim is best tested by showing that the SAME test behaves \