Cube brush on a binned source: the brushed trace owns the free-axis grid (P = its bins, not 2048 / 128)

[jvdd]

Status

• 2026-09-29: the histogram2d source is being fixed first on a worktree branch. Its free axis is now the source's own cell grid, p=(x_bins, y_bins), with zoomed axes snapped by snap_range. The 1-D histogram source follows with the same mechanism.
• Verified reachable on main: with x_bins=y_bins=200, a brush strictly inside one cell commits a box 1.56 cells wide per axis. Linked figures filter 27 rows where the cell holds 12. With the default 20x20 grid, a brush inside cell [30, 35) commits x = [31.25, 34.37), and linked figures show 471 rows instead of 964.

Summary

A histogram or histogram2d source brushes on a fixed grid (P = 2048, or 128 per axis) that ignores its bars and cells. So the highlighted bars, the filtered rows and the cube size all come out wrong. The fix is to let the brushed trace own the grid, as the target side already does.

Where P lives

Place	What it does
flexviz/cube.py:155	FreeAxisSpec.p defaults to 2048.
flexviz/trace/hist.py:251-267	Histogram source: p=2048, domain = raw viewport, or None unzoomed.
flexviz/trace/hist2d.py:302-336	Histogram2d source: kind="box2d", p=128 per axis.
flexviz/trace/base.py:586-610	Line and box sources (_range_cube_source_spec): p=2048.
flexviz/engine.py:726-731	Unzoomed free domain = raw column min/max, no pad.
flexviz/cube.py:681-689, 1292-1302	Free bin rule: natural floor, no round epsilon, degenerate top bin P.
flexviz/cube.py:107-121	day_grid: whole-day bins for a Date free axis.
flexviz/adapters/js/runtime/cube.js:27-28	Client copies: _FV_CUBE_P = 2048, _FV_CUBE_BOX2D_P = 128.
runtime/cube.js:500-512	fvCubeSnap: floor both ends, commit [edge(lo), edge(hi + 1)).
plotly/events.js:836-843, 913-918	Gesture snap, 1-D and box2d.
plotly/events.js:1534-1570	Commit override: snapped closed="left" predicate and redrawn box.
plotly/events.js:1710-1760	Echo guard: tolerance is half a snap bin.

The display grid lives elsewhere. snap_range (bin_grid.py:28-45) snaps a zoomed viewport to a lattice, with bins or bins + 1 bars. _histogram_bounds_exprs pads the top by _HIST_BIN_EPSILON = 1e-10 (hist.py:66, 434, 451). The Rust kernel adds a 1e-9 round epsilon and clamps the top (flexviz_polars/src/expressions.rs:764, 808). The delta sends [lo, step, n] per axis (hist.py:492, hist2d.py:480-481). The client keeps it in hoverEdgesByTraceUid (runtime/state.js:134).

The target side already uses the trace grid. Histogram.get_cube_target_spec returns bins and the snapped viewport (hist.py:296-313). The engine pads it like the display (engine.py:787). Histogram2d targets use x_bins and y_bins (hist2d.py:391-406). So the mismatch is on the source side only.

Which sources are affected

Trace	P today	Own grid?	Verdict
histogram	2048 (hist.py:267)	Yes: bins bars	Wrong. Must brush on its bars.
histogram2d	128 × 128 (hist2d.py:333)	Yes: x_bins × y_bins cells	Wrong. Must brush on its cells.
line	2048 (line.py:524)	Buckets exist (n_points // 2), but no edge is drawn	Fine. Keep 2048.
box	2048 (box.py:127)	No	Fine. Keep 2048.
bar, pie, treemap	none, categorical (bar.py:186, pie.py:145, treemap.py:92)	Exact labels	Not affected.
geo_histogram2d	not a cube source (geo_hist2d.py:151-157, kind geo_box)	Yes: cells	Not affected today. The contract below applies if it becomes a source.
corr_heatmap	not a source (base default kind none, base.py:171-181)	n/a	Not affected.
geo_line	not a source (geo_line.py:152-153)	n/a	Not affected.

What goes wrong today

1. Highlight and filter disagree (histogram)

Plotly selects a bar when its centroid is inside the box (plotly.js/src/traces/bar/select.js, getCentroid). For a vertical bar, the centroid is the bar center on x and the bar top on y. The figure brushes a full-height band (selectdirection="h", plotly_adapter.py:80-109). So only the center counts. No FlexViz code changes this rule.

Example: bins=20 on [0, 100], brush 13.1..47.2. The commit is [13.0859375, 47.216796875). The redrawn box highlights bars 3..8. The filter also takes 38 % of bar 2 and 44 % of bar 9. Both bars are drawn as unselected.

The drag has the same problem. Plotly highlights by centroid, while the cube slices 2048 bins.

2. The two grids line up only by chance

With bins=20, one bar is 102.4 P bins wide. Only inner edges 5, 10 and 15 (of 19) fall on a P edge. A brush cannot end on the other 16 bar edges.

3. The cube is 50 to 70 times too large

Measured with build_cube + encode_fvcube on 2M rows, free axis over the source column:

target	cells P=2048 → P=20	bytes P=2048 → P=20
hist, 20 bins	18,875 → 305	227 KB → 4.2 KB
grouped hist, 5 groups	73,506 → 1,316	1.18 MB → 21.7 KB
hist2d, 20×20	134,851 → 3,201	2.16 MB → 51.9 KB

4. Histogram2d: the box cuts through cells

Plotly heatmaps have no selection highlight. The heatmap module has no selectPoints. So the visible problem is the redrawn box, which does not follow cell edges. With the default 20 × 20 grid (figure.py:459-460), one cell is 6.4 P bins wide. Edge cells are partly filtered. Above 128 bins per axis, one P bin is wider than one cell. Then no brush can select a single cell.

5. Zoomed source: rows in the edge bars cannot be selected

A zoomed histogram snaps its viewport outward to the lattice (bin_grid.py:28-45). The mask keeps the whole snapped span (bin_grid.py:74-80). So the first and last bars count rows outside the viewport. The free domain is the raw viewport (hist.py:267, via engine.py:539). The build drops rows outside it (cube.py:1299). The snap clamps at bin 0 (cube.js:503). So the commit never reaches below the viewport. These rows are counted in a bar, but no brush can select them. This is up to one bar width on each side. Histogram2d has the same problem per axis (engine.py:602-607).

6. Small, all range sources: a zoomed commit goes one P bin past the viewport

A brush past the viewport top lands in the degenerate bin P. The commit then ends at lo + (P + 1)·step (cube.js:510). The cube has no rows there (cube.py:1299). So the live slice and the restored predicate differ for rows in (hi, hi + span/P). tests/test_browser_cube.py:4337 pins this bound.

What is NOT a problem

• Line source. The drawn points are min/max extrema at their real x. No bucket edge is visible. At P = 2048, one snap step is about half a pixel on a 1000 px plot. The line buckets (500 by default, figure.py:191) are coarser and also invisible. Keep 2048.
• Box source. A box has no grid on its data axis. Keep 2048.
• Categorical sources. Bar, pie and treemap select exact labels. They have no P.
• Target side. Histogram and histogram2d targets already bin on the trace grid. The line target has its own grid mismatch, tracked in Line envelope: cube grid and trace grid place bucket edges differently #24.
• Cube reuse across sources. Today a histogram source and a line source on one column can share a cube. After the change they cannot. That only costs a rebuild. The client key carries p (cube.js:47-72), so it cannot mix two grids.
• Grouped histograms with barmode="group". Child bars sit inside their bin, off center. During the drag, Plotly can highlight only part of an edge bin. After the commit, the box sits on bin edges and holds every child centroid. So the committed state agrees.

Proposed contract

The brushed trace owns the free-axis grid. A binned source brushes on its own display grid. A source without a grid keeps P = 2048.

1. get_cube_source_spec returns the same (P, domain) as the trace's binned target dim. Histogram: bins, and the viewport snapped by snap_range (None unzoomed). Histogram2d: (x_bins, y_bins), snapped per axis. Line and box: 2048 and the raw viewport, as today.
2. The engine resolves the free domain with the target-dim code (_resolved_target_dims, engine.py:751-790). A 1-D free axis gets the 1e-10 pad. A box2d axis gets none. This is exactly what the display does.
3. Every range free axis bins with the kernel rule, _fixed_hist_bin_expr (cube.py:726-744). It adds the 1e-9 round epsilon and clamps the top. The natural floor and the degenerate top bin go.
4. The client snaps each brush end to the nearest grid edge. For bars, this is exactly Plotly's centroid rule. So the drag highlight, the cube slice and the commit agree.
5. The commit carries the kernel's own bin boundaries: [lo + (k − ε)·step, lo + (m + 1 − ε)·step), with ε = 1e-9. When bar m is the last bar, the top is lo + n·step with closed="both". Temporal edges then ceil to the unit, as fvPhysicalToTemporal does today (cube.js:160-164).
6. The client reads a binned source's grid from the cube header (free.p, free.domain), which the server resolves. Until the first header arrives, a commit stays unsnapped and goes through the normal POST, which is still correct. The edge triple in hoverEdgesByTraceUid is not used: it is in epoch-ms for temporal axes, lo + n*step can differ from the server's hi by one ulp, and it can be stale while a zoom delta is in flight. No spec field and no wire field are added.

closed="both" at the top is needed. The pad vanishes above about 1e6, because 1e7 + 1e-10 == 1e7 in float64. So [.., hi + pad) can drop the rows at the maximum.

Hard cases and the rule for each

Zoom. Real. The grid changes with each viewport, with bins or bins + 1 bars. Rule: the source spec snaps exactly as the target spec does (hist.py:299-303). The delta triple follows the same snap. This also fixes item 5.

Rows on a bar edge. Real for integer data. The raw bar edge 25.000000000025 excludes the value 25, but the kernel puts 25 in the bar that starts there. Rule: contract item 5. Checked on 0..100, bins=20: raw edges put the wrong side at 19 of 19 inner edges. Edges with ε put it right at all 19. Integer columns round the bound inward (base.py:745-768), so they agree too.

Bin rule on the new grid. Real. With P = bins, the current natural floor puts 19 of the 101 values 0..100 one bin lower than the kernel. Rule: contract item 3.

Date axes (unit="day"). Real. Bar edges are fractional days. day_grid must not re-grid a histogram source. For 365 days and 20 bars, it gives 19-day bins under 18.25-day bars. Rule: ceil the kernel boundary, not the raw edge. Raw ceil(edge) is wrong at 4 of 19 edges (365 days) and 19 of 19 edges (100 days). Ceil with ε is right at all of them.

Histogram2d with many bins. Real only above 128 bins per axis. With the kernel rule, the composite key is nx·ny, not (nx+1)(ny+1). The default 20 × 20 grid gives 400 free bins instead of 16,641. A 500 × 500 grid gives 250,000, plus 1 MB of CSR offsets on the client (cube.js:436-437). No cube cell cap exists (Architecture.md:903, #19). The client store holds 256 MB (cube.js:29), the server cache 512 MB (cache.py:106). See open decision 2.

Two range traces on one column in one figure. Real, but rare. Today the last trace in spec order wins (events.js:506-509). The server takes P from the trace the client names (engine.py:539). Rule: the client names a binned trace when there is one. The server takes P from that trace. This needs no wire change.

Cube cache shared by a histogram source and a line or box source. Not a problem. See above.

Echo tolerance. Real, small. The guard reads _FV_CUBE_P (events.js:1739). Rule: the tolerance is half the source step. With the nearest-edge snap, any range inside that radius snaps to the same edges. So dropping it is exact. With today's floor snap, it is only close to exact.

Brush inside one bar. Real. Both ends snap to the same edge, so no bar is selected. Plotly highlights no bar either. See open decision 3.

Open decisions

1. One bin rule for every range source. Recommended: yes (contract item 3). Line and box then lose the degenerate top bin, and day_grid is no longer needed. Ceil with ε is exact for whole-day data on any grid. The other option keeps the natural floor for line and box. That needs a second bin rule, picked per source.
2. Histogram2d above 128 bins per axis. No cap, in line with Support dense Hist2D and high-cardinality grouped traces out of core #19, or a cell budget with a fallback to mouseup-only. Recommended: no cap. The source delta already sends nx·ny values.
3. A brush that covers no bar center. Commit a clear, or select the bar that holds the brush. Recommended: a clear, because Plotly highlights nothing.
4. The 1e-10 pad. Keep it, and pad the free axis too (contract item 2). Or delete it everywhere. The kernel top clamp already puts the maximum in the last bar, and above about 1e6 the pad does nothing. Deleting it also removes bin_variant. Recommended: keep it here, and track the deletion on its own.

Spec and wire impact

• Spec: no change. A predicate keeps {column, range, closed}. closed="both" exists already (spec.py:117-125). Share URLs keep their shape, only the edge values change. No spec version bump.
• FVCube header. The free block keeps {kind, p, domain, unit} (cube.py:1503-1514). For box2d, p becomes a pair when x_bins != y_bins, like unit. Client and server ship together, so no compat path is needed. w and p_eff go if day_grid goes.
• Cube content key. Bump its v (cube.py:1782) if the bin rule changes. A line source keeps p=2048 and its domain, so its key would stay equal while the blob changes. CacheBackend is a protocol (cache.py:31), so a stored blob can outlive a restart.

Tests to change

• tests/test_trace_hist.py:850-870: source p and domain (bins, snapped viewport).
• Histogram2d source tests in tests/test_cube.py and tests/test_cube_server.py: per-axis p.
• tests/test_cube.py: degenerate-top-bin tests (265-275, 1122-1135, 1709-1715) move to the top clamp. day_grid tests (1633 on) go if decision 1 lands.
• tests/test_browser_cube.py: _P (102) and the snapped-edge asserts (499-510, 542-551, 688-699, 4231-4235, 4337, 4527-4529).
• New regression tests. Each fails on main today.
  • Browser: a brush over bars k..m commits their kernel boundaries. The highlighted bars equal the filtered bars.
  • Integer data on bar edges (0..100, bins=20): filtered rows equal the rows in the highlighted bars. Same for a Date column.
  • A zoomed histogram source can select the off-screen rows of its edge bars.
  • A zoomed brush past the grid top selects no row above the grid.
  • The echo guard drops a re-emitted range within half a bar. No test covers the echo guard today.
  • tests/test_perf_choices.py: the cube cells of a histogram source scale with bins, not with 2048.
• Architecture.md: 1072-1082 (P = 2048), 1164-1166 (snap), 1281-1284 (day grid), 1322-1324 (support table), 1338-1341 (sources table).

Related issues

• Line envelope: cube grid and trace grid place bucket edges differently #24: the line target grid differs from the line trace grid. Same family, target side.
• Zoomed histogram grid: nice-step bin widths with a target bin count #36: nice-step zoomed bin widths. The source grid follows with no extra work.
• A zoomed bin grid does not match the unzoomed grid when the viewport covers the data #124: the zoomed grid differs from the unzoomed grid. The source grid follows with no extra work.
• Let the bin grid travel with the aggregation result #37: let the bin grid travel with the aggregation result. The source spec needs the grid without the aggregation.
• Cube live brush draws temporal histogram targets in the physical unit, not epoch-ms #128: temporal histogram targets drawn in the physical unit. The same epoch-ms versus physical conversion.
• Support dense Hist2D and high-cardinality grouped traces out of core #19: no output cell cap. Bears on open decision 2.
• Log axes: a zoom re-aggregates the wrong range, and bins and line buckets ignore the log scale #98: log axes. The cube grid ignores the axis scale.

[jvdd]

Fixed by #134 (merged 2026-10-01, 515d7e6). The PR said "Refs", so the issue did not close on merge.

What landed:

• A histogram source brushes on its own bars (p = bins, viewport snapped with snapped_domain). A histogram2d source brushes on its own cells (p = (x_bins, y_bins)). Line and box keep p = 2048.
• One bin rule on every range free axis: _fixed_hist_bin_expr, also in the line-envelope kernel. day_grid is removed.
• A brush selects each bin whose drawn center it covers, ends included. The commit writes the kernel's bin boundaries, so the highlighted bars and the filtered rows agree.

Open decisions:

1. One bin rule for all range sources: yes.
2. No cell cap for histogram2d (see Support dense Hist2D and high-cardinality grouped traces out of core #19).
3. A brush that covers no bin center clears the selection.
4. The 1e-10 pad is deleted, not kept. The kernels' top clamp covers it.

Tracked elsewhere: edge bars of a zoomed source (#124), the cell cap (#19), time-zone-aware sources (#139), the zoomed line-envelope prefilter (#140).

Release: the next flexviz release must ship flexviz-polars and raise the pin, because the line-envelope kernel changed (CHANGELOG, Unreleased).