Current behavior
ABS_TOL_DEFAULT = 1e-08 and REL_TOL_DEFAULT = 1e-05 (diffly/_utils.py) are the defaults for compare_frames, the CLI, and diffly.testing.assert_frame_equal.
These match polars.testing.assert_frame_equal (which inherited them from pandas/NumPy). They do not match pl.Expr.is_close, which compare_frames calls under the hood and which defaults to rel_tol=1e-09, abs_tol=0.0 (same as math.isclose, which the compare_frames docstring says the implementation mirrors).
(The tolerances guide currently says the defaults match math.isclose; that's a separate docs fix — PR #58.)
Problem
The polars.testing parity makes sense for diffly.testing.assert_frame_equal — it's a drop-in replacement. But compare_frames is a diff tool, where the cost asymmetry is reversed: a test assertion biases loose to avoid flaky failures, while a diff report should bias toward flagging, since users only inspect what's flagged. With rel_tol=1e-05, a $1 change on a $100,000 value is hidden by default. abs_tol=1e-08 also bakes in a magnitude assumption that a general tool can't justify.
Proposal
Decouple the defaults:
compare_frames and the CLI: rel_tol=1e-09, abs_tol=0.0 — matching pl.Expr.is_close and math.isclose.diffly.testing.assert_frame_equal: keep 1e-05 / 1e-08 for polars.testing parity.
The tolerances guide's example (differences at the 10th decimal place, ~3e-11 relative) still passes under the stricter defaults.
Trade-offs
abs_tol=0.0 flags near-zero values that differ only by cancellation noise (e.g. 0.0 vs 1e-17). I think that's the correct behavior for a diff tool — surface it, let the user set abs_tol for their data's scale — but it will be visible to some users on upgrade.compare_frames(a, b).equal() could be False where diffly.testing.assert_frame_equal(a, b) passes. Documenting the two defaults side by side would mitigate this.- It's a behavior change for
compare_frames users relying on the current defaults.
If shared defaults are intentional (e.g. so a comparison always explains an assertion failure), a short note in the docs to that effect would resolve this too.
Current behavior
ABS_TOL_DEFAULT = 1e-08andREL_TOL_DEFAULT = 1e-05(diffly/_utils.py) are the defaults forcompare_frames, the CLI, anddiffly.testing.assert_frame_equal.These match
polars.testing.assert_frame_equal(which inherited them from pandas/NumPy). They do not matchpl.Expr.is_close, whichcompare_framescalls under the hood and which defaults torel_tol=1e-09, abs_tol=0.0(same asmath.isclose, which thecompare_framesdocstring says the implementation mirrors).(The tolerances guide currently says the defaults match
math.isclose; that's a separate docs fix — PR #58.)Problem
The
polars.testingparity makes sense fordiffly.testing.assert_frame_equal— it's a drop-in replacement. Butcompare_framesis a diff tool, where the cost asymmetry is reversed: a test assertion biases loose to avoid flaky failures, while a diff report should bias toward flagging, since users only inspect what's flagged. Withrel_tol=1e-05, a $1 change on a $100,000 value is hidden by default.abs_tol=1e-08also bakes in a magnitude assumption that a general tool can't justify.Proposal
Decouple the defaults:
compare_framesand the CLI:rel_tol=1e-09, abs_tol=0.0— matchingpl.Expr.is_closeandmath.isclose.diffly.testing.assert_frame_equal: keep1e-05/1e-08forpolars.testingparity.The tolerances guide's example (differences at the 10th decimal place, ~3e-11 relative) still passes under the stricter defaults.
Trade-offs
abs_tol=0.0flags near-zero values that differ only by cancellation noise (e.g.0.0vs1e-17). I think that's the correct behavior for a diff tool — surface it, let the user setabs_tolfor their data's scale — but it will be visible to some users on upgrade.compare_frames(a, b).equal()could beFalsewherediffly.testing.assert_frame_equal(a, b)passes. Documenting the two defaults side by side would mitigate this.compare_framesusers relying on the current defaults.If shared defaults are intentional (e.g. so a comparison always explains an assertion failure), a short note in the docs to that effect would resolve this too.