diff --git a/CLAUDE.md b/CLAUDE.md index 7536204..ee16581 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1526,6 +1526,68 @@ of it**, measured on trees verified identical beforehand. What made that safe to it was that the README transcripts and the figure gate would have said otherwise if it were not. +### `strict=` is the whole point, and ruff's own fix takes it away + +`zip()` without `strict=` truncates to the shorter argument and says nothing. +The ratchet recorded 41, and the obvious move is `ruff --fix --unsafe-fixes`. +**It writes `strict=False`** -- the current behaviour, spelled out loud. That +takes the count to zero, looks exactly like the job being done, and preserves +every defect the rule exists to expose. Check *which* keyword a mechanical fix +inserted before believing the number it produced. + +Done by hand instead, and the count was the least interesting part: + +- **58 sites, not 41.** The other 17 are in `notebooks/`, which is not in + `SOURCE_DIRS` in `tests/test_lint_debt_only_shrinks.py`, so no ruff guard here + had ever read them -- in the three notebooks a student is most likely to open. + Same shape as `KNOWN_NON_EXAMPLE_CHAPTER_FILES`: **a file outside the sweep's + scope is a file nothing sweeps**, and that scope is a tuple somebody wrote + once. +- **One real defect**, in `ch5_fingerprinting/example_comparison.py`, where + `colors_box` held five shades against six methods. The sixth box kept + matplotlib's default blue among five pastels and read as deliberately + highlighted. Nothing raised, nothing left a range, no printed number moved -- + the sixth box simply never got its colour, which only opening the PNG shows. +- **One site where `strict=True` is wrong.** `zip(scales, scales[1:])` is the + consecutive-pairs idiom and *depends* on the truncation, so strict raises on + every call; it is `itertools.pairwise` now. `zip(xs, itertools.count())` and + `zip(xs, cycle(ys))` are the same family. **The tell is a second argument + derived from the first**, and a grep for `[1:]` finds only the spelling you + thought of -- read the hunks. + +**The entry is deleted from BASELINE rather than set to 0, and the deletion is +the guard**: `test_no_rule_has_more_findings_than_it_did` fails on any rule not +listed, so one new unguarded `zip()` turns it red. Verified by adding one and +watching `B905: 1 findings, new`, not by reading the assertion. + +**A green sweep only means something where the line runs, so that was measured +too. 41 of the 42 `.py` sites execute**, and the one that does not is the +interesting entry. 34 run under the examples, the generators and two flag-gated +paths a plain run never reaches -- ch4's `--data` branch and ch7's `--animate` +callback, which between them hold two sites nothing else touches. Six more sit +inside test bodies and one in `core/fingerprinting`, all seven confirmed rather +than assumed. The last, ch8's unobservable-mode print, is behind +`if n_unobservable > 0`, which the fixes-aided configuration never enters. It +is equal-length by construction -- `Vt` from `svd(..., full_matrices=True)` is +square, so a mode column is `n_states` long and `state_names` comes from the +same analysis -- and its identical twin in the odometry-only branch executes. +The 17 notebook sites are covered by `tests/docs/test_notebooks_run.py`. + +**The measurement lied twice before it said anything true**, both times looking +exactly like "this code never runs": + +- `[tool.coverage.run] source = ["ipin_examples", "core"]` in `pyproject.toml` + means a plain `coverage run` over a chapter example measures `core/` and + **not the example**, so the first cross-reference reported 42 of 43 sites as + never loaded. Pass `--source` explicitly when measuring something the project + does not normally measure. +- `coverage combine` **starts clean unless you pass `--append`**. Running it a + second time discarded a finished 16-step sweep, and the same script then + reported 1 site executed where it had reported 32. + +Both were caught the same way: a number moved while the code did not. That is +the fifth harness in this file to report the thing it could not read as broken. + ## Parallel sessions Several agents often work this repo at once, on separate worktrees off `main`. diff --git a/ch3_estimators/example_comparison.py b/ch3_estimators/example_comparison.py index 4bb3b1a..5fcfd33 100644 --- a/ch3_estimators/example_comparison.py +++ b/ch3_estimators/example_comparison.py @@ -656,7 +656,7 @@ def main(): ax.grid(True, alpha=0.3, axis="y") # Add value labels on bars - for bar, n_evals in zip(bars, evaluations): + for bar, n_evals in zip(bars, evaluations, strict=True): height = bar.get_height() ax.text( bar.get_x() + bar.get_width() / 2.0, diff --git a/ch4_rf_point_positioning/example_aoa_positioning.py b/ch4_rf_point_positioning/example_aoa_positioning.py index a207a80..af80e5f 100644 --- a/ch4_rf_point_positioning/example_aoa_positioning.py +++ b/ch4_rf_point_positioning/example_aoa_positioning.py @@ -386,7 +386,7 @@ def visualize_aoa_geometry(): # # Negating gives the anchor-to-agent direction, so the rays now meet at # the target the way an AOA geometry figure is meant to show. - for i, (anchor, psi) in enumerate(zip(anchors, aoa_noisy)): + for i, (anchor, psi) in enumerate(zip(anchors, aoa_noisy, strict=True)): # In ENU: psi is from North (+y), so the agent lies from the anchor at # E-component = -sin(psi), N-component = -cos(psi) line_length = 15 diff --git a/ch4_rf_point_positioning/example_comparison.py b/ch4_rf_point_positioning/example_comparison.py index 49c6a6a..82e9afa 100644 --- a/ch4_rf_point_positioning/example_comparison.py +++ b/ch4_rf_point_positioning/example_comparison.py @@ -778,7 +778,7 @@ def plot_dataset_results(results: Dict, output_file: str = None): results["gdop"]["AOA"], ] bp = ax3.boxplot(gdop_data, tick_labels=["TOA", "TDOA", "AOA"], patch_artist=True) - for patch, color in zip(bp["boxes"], ["blue", "red", "green"]): + for patch, color in zip(bp["boxes"], ["blue", "red", "green"], strict=True): patch.set_facecolor(color) patch.set_alpha(0.6) ax3.set_ylabel("GDOP") @@ -934,7 +934,7 @@ def plot_inline_comparison(noise_levels, results): # 1. RMSE vs Noise ax1 = axes[0, 0] - for method, color in zip(methods, colors): + for method, color in zip(methods, colors, strict=True): rmse_values = [ np.sqrt(np.mean(e**2)) if len(e) > 0 else np.nan for e in results[method] ] @@ -950,7 +950,7 @@ def plot_inline_comparison(noise_levels, results): # 2. Error CDF ax2 = axes[0, 1] noise_idx = 2 - for method, color in zip(methods, colors): + for method, color in zip(methods, colors, strict=True): errors = results[method][noise_idx] if len(errors) > 0: sorted_errors = np.sort(errors) @@ -968,7 +968,7 @@ def plot_inline_comparison(noise_levels, results): data = [results[m][noise_idx] for m in methods if len(results[m][noise_idx]) > 0] labels = [m for m in methods if len(results[m][noise_idx]) > 0] bp = ax3.boxplot(data, tick_labels=labels, patch_artist=True, showfliers=False) - for patch, color in zip(bp["boxes"], colors[: len(data)]): + for patch, color in zip(bp["boxes"], colors[: len(data)], strict=True): patch.set_facecolor(color) patch.set_alpha(0.6) ax3.set_ylabel("Position Error (m)") @@ -986,7 +986,7 @@ def plot_inline_comparison(noise_levels, results): ax4 = axes[1, 1] total_points = 50 dashes = ["-", "--", "-.", ":"] - for method, color, dash in zip(methods, colors, dashes): + for method, color, dash in zip(methods, colors, dashes, strict=True): rates = [len(e) / total_points * 100 for e in results[method]] ax4.plot( noise_levels, diff --git a/ch5_fingerprinting/example_classification.py b/ch5_fingerprinting/example_classification.py index 65c30bc..1f34efc 100644 --- a/ch5_fingerprinting/example_classification.py +++ b/ch5_fingerprinting/example_classification.py @@ -442,7 +442,7 @@ def evaluate_hierarchical_localization(db: FingerprintDatabase, rng=None): ax.set_title("RMSE Comparison", fontsize=12) ax.grid(True, alpha=0.3, axis="y") # Add values on bars - for bar, rmse in zip(bars, rmses): + for bar, rmse in zip(bars, rmses, strict=True): height = bar.get_height() ax.text( bar.get_x() + bar.get_width() / 2, diff --git a/ch5_fingerprinting/example_comparison.py b/ch5_fingerprinting/example_comparison.py index b8093a6..19b7ec6 100644 --- a/ch5_fingerprinting/example_comparison.py +++ b/ch5_fingerprinting/example_comparison.py @@ -289,7 +289,7 @@ def evaluate_scenario(scenario_name, db, queries, true_locs, floor_id=None): method_name = "NN (Euclidean)" print(f" {method_name}...", end=" ", flush=True) errors, times = [], [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = nn_localize(query, db, metric="euclidean", floor_id=floor_id) t_end = time.perf_counter() @@ -315,7 +315,7 @@ def evaluate_scenario(scenario_name, db, queries, true_locs, floor_id=None): method_name = "k-NN (k=3)" print(f" {method_name}...", end=" ", flush=True) errors, times = [], [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = knn_localize( query, @@ -356,7 +356,7 @@ def evaluate_scenario(scenario_name, db, queries, true_locs, floor_id=None): method_name = "MAP" print(f" {method_name}...", end=" ", flush=True) errors, times = [], [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = map_localize(query, model_bayes, floor_id=floor_id) t_end = time.perf_counter() @@ -382,7 +382,7 @@ def evaluate_scenario(scenario_name, db, queries, true_locs, floor_id=None): method_name = "Posterior Mean" print(f" {method_name}...", end=" ", flush=True) errors, times = [], [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = posterior_mean_localize(query, model_bayes, floor_id=floor_id) t_end = time.perf_counter() @@ -408,7 +408,7 @@ def evaluate_scenario(scenario_name, db, queries, true_locs, floor_id=None): method_name = "Post.Mean (k=10)" print(f" {method_name}...", end=" ", flush=True) errors, times = [], [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = posterior_mean_localize( query, model_bayes, floor_id=floor_id, top_k=10 @@ -443,7 +443,7 @@ def evaluate_scenario(scenario_name, db, queries, true_locs, floor_id=None): method_name = "Linear Regression" print(f" {method_name}...", end=" ", flush=True) errors, times = [], [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = model_lr.predict(query) t_end = time.perf_counter() @@ -604,8 +604,19 @@ def main(): ax4 = plt.subplot(3, 3, 4) error_data = [r["errors"] for r in all_results["Baseline"]] bp = ax4.boxplot(error_data, tick_labels=methods, patch_artist=True) - colors_box = ["lightblue", "lightcyan", "lightcoral", "lightsalmon", "lightgreen"] - for patch, color in zip(bp["boxes"], colors_box): + # One light shade per entry of `colors` above, in the same order. It was a + # shade short, so the sixth method's box kept matplotlib's default facecolor + # and read as deliberately highlighted among five pastel ones. strict=True + # below is what stops the two lists drifting apart again. + colors_box = [ + "lightblue", + "lightcyan", + "lightcoral", + "lightsalmon", + "lightgreen", + "plum", + ] + for patch, color in zip(bp["boxes"], colors_box, strict=True): patch.set_facecolor(color) ax4.set_ylabel("Positioning Error (m)") ax4.set_title("Error Distribution (Baseline)") @@ -641,7 +652,7 @@ def main(): # actual finding, so it should not be what breaks the labelling. Colours # match the per-query cost panel above, so a method reads the same in both. ax6 = plt.subplot(3, 3, 6) - for r, color in zip(all_results["Baseline"], colors): + for r, color in zip(all_results["Baseline"], colors, strict=True): ax6.scatter( r["ops_per_query"], r["rmse"], diff --git a/ch5_fingerprinting/example_deterministic.py b/ch5_fingerprinting/example_deterministic.py index 867bf5f..797f030 100644 --- a/ch5_fingerprinting/example_deterministic.py +++ b/ch5_fingerprinting/example_deterministic.py @@ -77,7 +77,7 @@ def generate_test_queries(db, n_queries=100, floor_id=None, noise_std=0.0, seed= # Generate fingerprints by interpolating from nearby RPs query_fingerprints = [] - for i, (true_loc, fid) in enumerate(zip(true_locs, floor_ids_out)): + for i, (true_loc, fid) in enumerate(zip(true_locs, floor_ids_out, strict=True)): # Find k nearest RPs for interpolation if floor_id is not None: dists = np.linalg.norm(rp_locs - true_loc, axis=1) @@ -171,7 +171,7 @@ def evaluate_positioning_method(method_name, method_fn, queries, true_locs, **kw errors = [] times = [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = method_fn(query, **kwargs) t_end = time.perf_counter() diff --git a/ch5_fingerprinting/example_probabilistic.py b/ch5_fingerprinting/example_probabilistic.py index 0feec44..2e2cd51 100644 --- a/ch5_fingerprinting/example_probabilistic.py +++ b/ch5_fingerprinting/example_probabilistic.py @@ -65,7 +65,7 @@ def generate_test_queries(db, n_queries=100, floor_id=None, noise_std=0.0, seed= query_fingerprints = [] - for true_loc, fid in zip(true_locs, floor_ids_out): + for true_loc, fid in zip(true_locs, floor_ids_out, strict=True): if floor_id is not None: dists = np.linalg.norm(rp_locs - true_loc, axis=1) else: @@ -99,7 +99,7 @@ def evaluate_method(method_name, method_fn, queries, true_locs, **kwargs): errors = [] times = [] - for query, true_loc in zip(queries, true_locs): + for query, true_loc in zip(queries, true_locs, strict=True): t_start = time.perf_counter() est_loc = method_fn(query, **kwargs) t_end = time.perf_counter() diff --git a/ch5_fingerprinting/figs/comparison_all_methods.pdf b/ch5_fingerprinting/figs/comparison_all_methods.pdf index 7bf550e..84fdd56 100644 Binary files a/ch5_fingerprinting/figs/comparison_all_methods.pdf and b/ch5_fingerprinting/figs/comparison_all_methods.pdf differ diff --git a/ch5_fingerprinting/figs/comparison_all_methods.png b/ch5_fingerprinting/figs/comparison_all_methods.png index d20edd4..3ae673e 100644 Binary files a/ch5_fingerprinting/figs/comparison_all_methods.png and b/ch5_fingerprinting/figs/comparison_all_methods.png differ diff --git a/ch5_fingerprinting/figs/comparison_all_methods.svg b/ch5_fingerprinting/figs/comparison_all_methods.svg index d609ba3..4902699 100644 --- a/ch5_fingerprinting/figs/comparison_all_methods.svg +++ b/ch5_fingerprinting/figs/comparison_all_methods.svg @@ -4176,7 +4176,7 @@ L 373.539534 421.360488 L 345.418162 421.360488 L 345.418162 464.100593 z -" clip-path="url(#p945a551099)" style="fill: #1f77b4; stroke: #000000; stroke-linejoin: miter"/> +" clip-path="url(#p945a551099)" style="fill: #dda0dd; stroke: #000000; stroke-linejoin: miter"/> ECEF...") ecef = np.array( - [llh_to_ecef(lat, lon, h) for lat, lon, h in zip(lats, lons, heights)] + [ + llh_to_ecef(lat, lon, h) + for lat, lon, h in zip(lats, lons, heights, strict=True) + ] ) print( f" ECEF X range: {ecef[:, 0].min()/1e3:.1f}km to {ecef[:, 0].max()/1e3:.1f}km" diff --git a/scripts/generate_ch6_env_sensors_dataset.py b/scripts/generate_ch6_env_sensors_dataset.py index 655b842..85dd650 100644 --- a/scripts/generate_ch6_env_sensors_dataset.py +++ b/scripts/generate_ch6_env_sensors_dataset.py @@ -567,7 +567,12 @@ def generate_dataset( # and got it wrong -- duplicated policy only has to be forgotten once. heading_true = att_true[:, 2] heading_error = np.abs( - np.array([wrap_angle_diff(e, t_) for e, t_ in zip(heading_est, heading_true)]) + np.array( + [ + wrap_angle_diff(e, t_) + for e, t_ in zip(heading_est, heading_true, strict=True) + ] + ) ) heading_error_deg = np.rad2deg(heading_error) mean_heading_error = np.mean(heading_error_deg) diff --git a/tests/ch6_dead_reckoning/test_comparison_figures.py b/tests/ch6_dead_reckoning/test_comparison_figures.py index ad02f67..4146a4a 100644 --- a/tests/ch6_dead_reckoning/test_comparison_figures.py +++ b/tests/ch6_dead_reckoning/test_comparison_figures.py @@ -70,7 +70,9 @@ def test_same_seed_gives_identical_measurements(self): first = self._noise(DEFAULT_SEED) second = self._noise(DEFAULT_SEED) - for label, a, b in zip(("accel", "gyro", "mag", "wheel"), first, second): + for label, a, b in zip( + ("accel", "gyro", "mag", "wheel"), first, second, strict=True + ): with self.subTest(sensor=label): np.testing.assert_array_equal(a, b) diff --git a/tests/ch6_dead_reckoning/test_heading_error_is_wrapped.py b/tests/ch6_dead_reckoning/test_heading_error_is_wrapped.py index 67da326..203c3f1 100644 --- a/tests/ch6_dead_reckoning/test_heading_error_is_wrapped.py +++ b/tests/ch6_dead_reckoning/test_heading_error_is_wrapped.py @@ -67,7 +67,10 @@ def setUpClass(cls): ) cls.error = np.abs( np.array( - [wrap_angle_diff(e, y) for e, y in zip(cls.heading_est, cls.yaw_true)] + [ + wrap_angle_diff(e, y) + for e, y in zip(cls.heading_est, cls.yaw_true, strict=True) + ] ) ) diff --git a/tests/ch7_slam/test_slam_frontend_figure.py b/tests/ch7_slam/test_slam_frontend_figure.py index f6dc774..e7b72f8 100644 --- a/tests/ch7_slam/test_slam_frontend_figure.py +++ b/tests/ch7_slam/test_slam_frontend_figure.py @@ -232,7 +232,7 @@ def test_writing_the_figure_twice_is_byte_identical(self): first = save_figure(fig, Path(tmp) / "a", FIGURE_NAME) second = save_figure(fig, Path(tmp) / "b", FIGURE_NAME) - for lhs, rhs in zip(first, second): + for lhs, rhs in zip(first, second, strict=True): with self.subTest(fmt=lhs.suffix): self.assertEqual(lhs.read_bytes(), rhs.read_bytes()) finally: diff --git a/tests/core/eval/test_plots.py b/tests/core/eval/test_plots.py index 6318c8e..516e15e 100644 --- a/tests/core/eval/test_plots.py +++ b/tests/core/eval/test_plots.py @@ -216,7 +216,7 @@ def test_output_is_byte_reproducible(self, tmp_path): finally: plt.close(fig) - for lhs, rhs in zip(first, second): + for lhs, rhs in zip(first, second, strict=True): assert ( lhs.read_bytes() == rhs.read_bytes() ), f"{lhs.suffix} output is not reproducible" diff --git a/tests/core/fusion/test_fusion_tuning.py b/tests/core/fusion/test_fusion_tuning.py index 25578af..953a435 100644 --- a/tests/core/fusion/test_fusion_tuning.py +++ b/tests/core/fusion/test_fusion_tuning.py @@ -9,6 +9,7 @@ import unittest import warnings +from itertools import pairwise import numpy as np @@ -527,7 +528,9 @@ def test_scale_grows_monotonically_with_the_residual(self): cauchy_scales = [cauchy_R_scale(r, 2.385) for r in self.RESIDUALS] for scales in (huber_scales, cauchy_scales): - for earlier, later in zip(scales, scales[1:]): + # pairwise, not zip(scales, scales[1:]): that idiom relies on + # zip truncating, so strict=True raises on every call. + for earlier, later in pairwise(scales): self.assertLessEqual(earlier, later + 1e-12) def test_literal_printed_form_would_trust_outliers_more(self): diff --git a/tests/core/slam/test_submap_2d.py b/tests/core/slam/test_submap_2d.py index e7ecec8..c6845ba 100644 --- a/tests/core/slam/test_submap_2d.py +++ b/tests/core/slam/test_submap_2d.py @@ -277,7 +277,7 @@ def test_build_submap_from_trajectory(self): np.array([[3.0, 0.0]]), # From pose 2 ] - for pose, scan in zip(poses, scans): + for pose, scan in zip(poses, scans, strict=True): submap.add_scan(pose, scan) self.assertEqual(submap.n_scans, 3) diff --git a/tests/docs/test_readme_example_output.py b/tests/docs/test_readme_example_output.py index ce35b1f..1fe1319 100644 --- a/tests/docs/test_readme_example_output.py +++ b/tests/docs/test_readme_example_output.py @@ -172,7 +172,7 @@ def _matches(expected: str, actual: str) -> bool: got = actual.split(" ") if len(want) != len(got): return False - return all(_token_matches(w, g) for w, g in zip(want, got)) + return all(_token_matches(w, g) for w, g in zip(want, got, strict=True)) def _find_from(live, expected, start): diff --git a/tests/test_lint_debt_only_shrinks.py b/tests/test_lint_debt_only_shrinks.py index c0bab41..11dc000 100644 --- a/tests/test_lint_debt_only_shrinks.py +++ b/tests/test_lint_debt_only_shrinks.py @@ -30,9 +30,9 @@ **What is left is mostly not lint at all.** 727 of the remaining findings are UP006/UP045/UP035/UP007 -- `List[int]` for `list[int]`, `Optional[X]` for `X | None`. Those became legal only when the floor moved to 3.10, they are -mechanical, and they are worth doing in their own change. The ~140 after that -are the ones with actual content: 41 `zip()` calls without `strict=`, which -truncate silently to the shorter argument, are the interesting ones. +mechanical, and they are worth doing in their own change. The ~100 after that +are the ones with actual content -- B905 was the largest of them and is now +gone, audited rather than swept: see the comment on BASELINE below. Per-rule rather than a single total on purpose: a total lets ten fixed W293 pay for ten new B905, which is the opposite of what a ratchet is for. @@ -65,9 +65,16 @@ #: Findings per rule today. Only ever edit these downwards. #: #: UP0xx are the annotation modernisations that the 3.10 floor made available, -#: and are the bulk of what is left. B905 is the one group worth reading before -#: fixing: `zip()` without `strict=` truncates to the shorter argument without -#: saying so. The twelve remaining W293 sit inside argparse `epilog` strings, +#: and are the bulk of what is left. +#: +#: B905 is absent, and its absence is the guard: the `appeared` check below +#: fails on any rule not listed here, so one new `zip()` without `strict=` +#: turns this red. +#: The 41 it used to record were read one at a time rather than swept, which is +#: how the two that mattered were found -- a boxplot palette one shade short of +#: its methods, and one site where `strict=True` is simply wrong, because +#: `zip(xs, xs[1:])` relies on the truncation. The twelve remaining W293 sit +#: inside argparse `epilog` strings, #: where the whitespace is content that gets printed rather than layout -- #: black leaves those alone, correctly, and so should you. BASELINE = { @@ -75,7 +82,6 @@ "UP045": 178, "UP035": 123, "I001": 84, - "B905": 41, "UP007": 38, "B007": 28, "B028": 14,