Context
_build_line_env_cube in cube.py collects x, y and the free value of every row. Then fixed_line_envelope2d drops the rows outside the free domain. On a zoomed source, the free domain covers only part of the data, so the build collects many rows only to drop them again.
A filter free_val.is_between(lo, hi) before the collect avoids that work, but it costs one pass when the source is not zoomed. That is the common case.
Measurement
From the review of PR #134: release plugin, POLARS_MAX_THREADS=4, 8M rows, a 2048-bin free axis and 500 x buckets, medians of alternating runs.
| Wrapper |
Free domain = all data (39 runs) |
Free domain = 25 % of the data (15 runs) |
No filter (PR #134, as on main) |
44.5 ms |
28.8 ms |
is_between filter before the collect |
47.0 ms (1.06x) |
18.6 ms (0.65x) |
With the filter, the zoomed build also collects 4 times fewer rows. Each collected row holds three f64 columns.
Proposal
Filter only when the source is zoomed. The engine knows this: a zoomed source gets its domain from get_cube_source_spec, and an unzoomed source gets it from _resolve_cube_domains. For example, the engine can pass that to the build, or the build can compare the domain with the column's memoized min and max.
Add a zoomed case to benchmarks/test_codspeed_engine.py::test_build_line_cubes, so that CodSpeed tracks both paths.
Found in the review of PR #134.
Context
_build_line_env_cubeincube.pycollects x, y and the free value of every row. Thenfixed_line_envelope2ddrops the rows outside the free domain. On a zoomed source, the free domain covers only part of the data, so the build collects many rows only to drop them again.A filter
free_val.is_between(lo, hi)before the collect avoids that work, but it costs one pass when the source is not zoomed. That is the common case.Measurement
From the review of PR #134: release plugin,
POLARS_MAX_THREADS=4, 8M rows, a 2048-bin free axis and 500 x buckets, medians of alternating runs.main)is_betweenfilter before the collectWith the filter, the zoomed build also collects 4 times fewer rows. Each collected row holds three
f64columns.Proposal
Filter only when the source is zoomed. The engine knows this: a zoomed source gets its
domainfromget_cube_source_spec, and an unzoomed source gets it from_resolve_cube_domains. For example, the engine can pass that to the build, or the build can compare the domain with the column's memoized min and max.Add a zoomed case to
benchmarks/test_codspeed_engine.py::test_build_line_cubes, so that CodSpeed tracks both paths.Found in the review of PR #134.