feat: fitted trendline KPIs, no table chrome over rows that show, named empty IN values

CHANGELOG.md

@@ -7,8 +7,8 @@ while the major version is 0, minor bumps may include breaking changes and say s
 
 ### Upgrading from 0.5
 
-- A spec with a mixed chart's `kind: "area"`, a trendline's `rolling_type`, a big number's `date_format` or a heatmap's `x_order` or `y_order` is refused by chartwright 0.5.0 and earlier (`literal_error` or `extra_forbidden`); upgrade CI before committing one.
-- The design brain is version 9. `data.rolling-window-span` is a new warn-level rule, so `--design strict` can newly block a rolling trendline KPI whose time range can't hold its window. `advise --fix` fills `compare_suffix` as "vs prior 12 months" where a 12-step rolling window compares with the 12 steps before, and, with column types (`--profile`), `date_format` on a big number of a date column.
+- A spec with a mixed chart's `kind: "area"`, a trendline's `rolling_type` or `y_axis_truncate`, a big number's `date_format` or a heatmap's `x_order` or `y_order` is refused by chartwright 0.5.0 and earlier (`literal_error` or `extra_forbidden`); upgrade CI before committing one.
+- The design brain is version 10. `data.rolling-window-span` is a new warn-level rule, so `--design strict` can newly block a rolling trendline KPI whose time range can't hold its window. `advise --fix` fills `compare_suffix` as "vs prior 12 months" where a 12-step rolling window compares with the 12 steps before, and, with column types (`--profile`), `date_format` on a big number of a date column. It also removes a search box it filled on a table whose rows all show (below), so a spec it fixed before shows that change in its diff.
 
 ### Added
 
@@ -17,6 +17,12 @@ while the major version is 0, minor bumps may include breaking changes and say s
     - `advise` warns when the chart's time range, or a defaulted dashboard time filter it loads with, is too short to hold the window and the comparison (`data.rolling-window-span`), and apply's data check warns from the time buckets the chart really has.
 - A big number shown as a date: `"date_format": "%a %-d %b %Y"` on a `big_number_total` shows `MAX(updated_at)` as "Sat 3 Oct 2026", for a tile that says how fresh the data is. It works on a date or timestamp metric and on a number of epoch milliseconds, and replaces `number_format`. With column types (`advise --profile`), `advise --fix` fills it on a big number of a date column's `MIN` or `MAX`, which Superset shows by default as its day alone ("Tue 31").
 - Heatmap axis order: `x_order` (left to right) and `y_order` (top to bottom), each `a_to_z`, `z_to_a`, `value_asc` or `value_desc`. `"y_order": "a_to_z"` puts the first label on top, such as a cohort triangle's oldest cohort; omitted, the labels keep running A to Z from the bottom up. On Superset 6.1.0 a value order ranks each label by its total; older releases order by the cells, so `check` warns there. `decompile` reads the order back.
+- A trendline KPI fitted to its values: `"y_axis_truncate": true` draws the trendline from its lowest value to its highest instead of up from zero, so a trailing-12-month total that moves a few percent shows the movement rather than a flat line over a solid block. `decompile` reads Superset's unticked "Start y-axis at 0" back as it.
+- Apply's data check names the values a chart's `IN` filter lists that return no rows, on a column the chart groups or draws by: a line chart that lists four zones where two have data now says which two it draws nothing for. A series limit that drops values on purpose, or a time column, is left alone.
+
+### Changed
+
+- The design brain adds no page-size picker, pager or search box to a table whose rows all show in its panel. `advise --fix` now fills a search box only on a raw table that pages, where the box sits in the bar the page-size picker already draws, and removes one it filled on a table whose rows all show. `advise` names a `page_length` or `search_box` you wrote on such a table (`size.table-chrome`, info), since Superset draws a page-size picker for any page size, even over one page; set `page_length` to 0 or leave it out to show every row with no picker.
 
 ### Fixed
 

chartwright/compiler.py

@@ -236,7 +236,11 @@ def metric(m: str):
         p["x_axis"] = chart.time_column
         p["time_grain_sqla"] = chart.time_grain or DEFAULT_TIME_GRAIN
         p["show_trend_line"] = True
-        p["start_y_axis_at_zero"] = True
+        # "Start y-axis at 0" (BigNumberWithTrendline/controlPanel.tsx:96 at 4.1.4 and 5.0.0,
+        # :107 at 6.1.0, default true), drawn as the trendline's yAxis scale:
+        # !startYAxisAtZero (transformProps.ts:219 at 4.1.4 and 5.0.0, :326 at 6.1.0), so
+        # false fits the line to its values. Always written, true unless y_axis_truncate.
+        p["start_y_axis_at_zero"] = not chart.y_axis_truncate
         # The Advanced Analytics rolling window (BigNumberWithTrendline controlPanel.tsx
         # :179-228 at 4.1.4 and 5.0.0, :252-301 at 6.1.0), applied to the trendline by
         # rollingWindowOperator in buildQuery, so transformProps reads the number and

chartwright/decompile.py

@@ -668,6 +668,9 @@ def keep_row_limit() -> None:
         colour = _rgb_to_spec(p.get("color_picker"))
         if colour:
             out["trend_color"] = colour
+        # "Start y-axis at 0" unticked (an unset key is the control's default, true).
+        if p.get("start_y_axis_at_zero") is False:
+            out["y_axis_truncate"] = True
         _rolling_to_spec(p, out, losses, name)
     elif spec_type in ("timeseries_line", "timeseries_bar", "timeseries_area", "timeseries_scatter"):
         ms = [metric_one(m) for m in (p.get("metrics") or [])]
@@ -1019,7 +1022,8 @@ def keep_row_limit() -> None:
         mapped_here = mapped_here | _legend_to_spec(p, out, spec_type)
     if spec_type == "big_number_trend":
         mapped_here = mapped_here | {"compare_lag", "compare_suffix", "subtitle", "color_picker",
-                                     "rolling_type", "rolling_periods", "min_periods"}
+                                     "rolling_type", "rolling_periods", "min_periods",
+                                     "start_y_axis_at_zero"}
     if spec_type == "big_number_total":
         mapped_here = mapped_here | {"subtitle", "time_format", "force_timestamp_formatting"}
     if spec_type == "table":

chartwright/design/defaults.py

@@ -346,8 +346,8 @@ def _page_length(ctx: RuleContext, c):
 
 
 @_fill("default.search-box", "search_box", {"table"},
-       "a raw table of more than ~20 rows gets a search box, when its rows still fit "
-       "beside it (the 20 is judgement)",
+       "a raw table of more than ~20 rows that outgrow its panel, and so page, gets a "
+       "search box; one whose rows all show gets none (the 20 is judgement)",
        superset=False, superset_text="no search box",
        override="write search_box: false")
 def _search_box(ctx: RuleContext, c):
@@ -358,16 +358,21 @@ def _search_box(ctx: RuleContext, c):
     n = ctx.params.search_min_rows
     if c.row_limit <= n:
         return None, f"row_limit {c.row_limit} is {n} rows or fewer"
-    if not c.page_length:
-        # A paged table already draws the bar the search box sits in. On one page, the
-        # bar takes room from the rows, and must not push any behind the scrollbar.
-        h = ctx.height(c.name)
-        fits = math.floor(grid_rows_visible(h, table_header_units(controls=True)))
-        if c.row_limit > fits:
-            return None, (f"{c.row_limit} rows on one page at height {h:g}: a search bar "
-                          f"would leave room for {fits}; page the table or raise the height")
-    return True, (f"up to {c.row_limit} raw rows: searching beats scrolling "
-                  f"(the {n}-row threshold is judgement)")
+    h = ctx.height(c.name)
+    # The rows the panel shows with nothing above them. A table that shows every row
+    # takes no DataTables chrome (size.table-chrome): a search bar over rows the reader
+    # already sees only pushes the last ones behind the scrollbar.
+    shown = math.floor(grid_rows_visible(h, table_header_units()))
+    if c.row_limit <= shown:
+        return None, f"all {c.row_limit} rows show at height {h:g}"
+    if not c.page_length or c.page_length >= c.row_limit:
+        # The box sits in the bar a paged table already draws (its page-size picker), so
+        # it costs no row there. A table on one page that outgrows its panel is either
+        # still to be paged (default.page-length) or too short to page.
+        return None, (f"{c.row_limit} rows on one page outgrow height {h:g}; the box comes "
+                      f"with paging")
+    return True, (f"up to {c.row_limit} raw rows over pages of {c.page_length}: searching "
+                  f"beats paging (the {n}-row threshold is judgement)")
 
 
 # -- legends and labels -------------------------------------------------------------

chartwright/design/guidelines/chart-choice.md

@@ -33,10 +33,16 @@
 - A rolling KPI (`rolling_type`) shows a trailing window, not the latest
   period: name the window in its title or subtitle ("Revenue, trailing 12
   months"), and give it a range that holds the window plus `compare_lag`
-  (24 months for a trailing year against the year before).
+  (24 months for a trailing year against the year before). Its trendline,
+  drawn up from zero, barely moves; when asked to show the movement, set
+  `y_axis_truncate`.
 - A paged table (`page_length`) needs room for one page and its pager, not
-  for every row; a timeseries with many groups keeps the top few with
+  for every row; a table whose rows all show takes no `page_length` or
+  `search_box`. A timeseries with many groups keeps the top few with
   `series_limit`.
+- Superset draws every line of a mixed chart solid at one width; draw a
+  reference series lighter with a pale `label_colors` colour in its own
+  query.
 - One dominant category flattening its siblings is the data talking: note it
   or filter it, don't hide it.
 - Every chart with a time axis should tolerate the dashboard time filter;

chartwright/design/model.py

@@ -32,7 +32,11 @@
 # ordered by value alone; default.date-tile fills date_format on a big number of a date
 # column (data-aware), and a big number shown as a date gets no number format from
 # narrative.big-number-format, default.count-format or a standard.
-DESIGN_BRAIN_VERSION = "9"
+# "10" = no DataTables chrome on a table whose rows all show: default.search-box fills only
+# a paged table whose rows outgrow its panel (and --fix removes a box it filled on one
+# whose rows all show), and size.table-chrome, a new info rule, names an author's page
+# size or search box there.
+DESIGN_BRAIN_VERSION = "10"
 
 KPI_TYPES = {"big_number_total", "big_number_trend"}
 TIMESERIES_TYPES = {"timeseries_line", "timeseries_bar", "timeseries_area", "timeseries_scatter"}

chartwright/design/rules.py

@@ -27,6 +27,7 @@
     grid_rows_visible,
     grid_units_for_rows,
     row_items,
+    table_header_units,
     without_superset_defaults,
 )
 from .model import AXIS_TYPES, KPI_TYPES, TIMESERIES_TYPES, Finding, RuleContext, rule
@@ -201,6 +202,45 @@ def table_window(ctx: RuleContext):
             )
 
 
+@rule("size.table-chrome", "info",
+      "a table whose rows all show in its panel draws no page-size picker, pager or search "
+      "box", since="10")
+def table_chrome(ctx: RuleContext):
+    """DataTables chrome over rows the reader already sees. Any page_length above 0 draws
+    the page-size picker, on one page too (hasPagination, plugin-chart-table
+    DataTable/DataTable.tsx:123 at 4.1.4 and 5.0.0, :173 at 6.1.0), and the pager joins
+    it from a second page (:407 at 4.1.4 and 5.0.0, :616 at 6.1.0); search_box
+    (include_search) draws the search bar (:383 at 4.1.4 and 5.0.0, :593 at 6.1.0). The
+    brain's own fills never add them to such a table (default.page-length,
+    default.search-box), so this names the author's."""
+    for c in ctx.spec.charts:
+        if c.type != "table" or c.row_limit is None:
+            continue
+        h = ctx.height(c.name)
+        # The rows the panel shows with nothing above or below them (spec.py's grid model).
+        shown = math.floor(grid_rows_visible(h, table_header_units()))
+        if c.row_limit > shown:
+            continue  # the rows outgrow the panel: paging and search earn their room
+        chrome, fields = [], []
+        if c.page_length and not ctx.brain_owns(c, "page_length"):
+            pages = math.ceil(c.row_limit / c.page_length)
+            chrome.append("a page-size picker" + (f" and a pager over {pages} pages"
+                                                  if pages > 1 else ""))
+            fields.append(f"page_length {c.page_length}")
+        if c.search_box and not ctx.brain_owns(c, "search_box"):
+            chrome.append("a search box")
+            fields.append("search_box")
+        if not chrome:
+            continue
+        yield Finding(
+            "size.table-chrome", "info", c.name, ctx.where(c.name),
+            f"all {c.row_limit} rows show at height {h:g}, yet {' and '.join(fields)} "
+            f"draw{'s' if len(fields) == 1 else ''} {' and '.join(chrome)} over them; "
+            f"remove {'it' if len(fields) == 1 else 'them'} so the table reads whole "
+            f"(page_length 0 shows every row on one page with no picker)",
+        )
+
+
 @rule("size.hbar-window", "warn", "horizontal bars need ~0.5 units of height per bar", fixable=True)
 def hbar_window(ctx: RuleContext):
     for c in ctx.spec.charts:

chartwright/smoke.py

@@ -8,7 +8,8 @@
 stays the renderer's job.
 
 Pass = HTTP 200 AND non-empty result payload. Empty = warning naming the
-chart, not a silent pass.
+chart, not a silent pass; so are values an IN filter lists on a column the
+chart groups or draws by that return no rows (they would draw nothing).
 """
 
 from __future__ import annotations
@@ -79,6 +80,50 @@ def _window_warning(chart, rows: int) -> str | None:
             + (" and no change" if lag else "") + "; widen the chart's time range")
 
 
+def _value_key(v) -> str:
+    """A filter value or a returned one, compared as text: a list of [2, 4] matches the 2
+    and 4 a numeric column returns, and 2.0 reads as 2."""
+    if isinstance(v, bool):
+        return str(v).lower()
+    if isinstance(v, float) and v.is_integer():
+        return str(int(v))
+    return str(v)
+
+
+def _series_limited(chart, column: str) -> bool:
+    """The chart keeps only its top N series by `column`, so listed values it drops are
+    the limit's doing, not missing data."""
+    holders = (chart.a, chart.b) if chart.type == "mixed" else (chart,)
+    return any(getattr(h, "series_limit", None) and getattr(h, "groupby", None) == column
+               for h in holders)
+
+
+def _empty_values_warning(chart, queries: list[dict], result: list, ds) -> str | None:
+    """Values a chart lists with IN on a column it groups or draws by, which come back
+    with no rows: a line chart that names four zones where two have data draws two lines
+    and a legend of two, and nothing on the dashboard says the other two are missing.
+    Only a column the query returns can be checked, since the data says which values
+    came back; a time column comes back as timestamps, so it is left alone."""
+    notes = []
+    for f in chart.filters:
+        if f.op != "IN" or f.sql is not None or not isinstance(f.value, list):
+            continue
+        column = f.column
+        pairs = [(q, r) for q, r in zip(queries, result) if column in q.get("columns", [])]
+        if not pairs or ds.is_temporal(column) or _series_limited(chart, column):
+            continue
+        back = {_value_key(row.get(column)) for _, r in pairs for row in r.get("data") or []}
+        missing = [v for v in f.value if _value_key(v) not in back]
+        if not missing:
+            continue
+        capped = any(len(r.get("data") or []) >= q.get("row_limit", math.inf) for q, r in pairs)
+        names = ", ".join(repr(v) for v in missing)
+        notes.append(f"{column} IN lists {names}, which return{'s' if len(missing) == 1 else ''}"
+                     f" no rows" + (" within the row limit" if capped else "")
+                     + f", so the chart draws nothing for {'it' if len(missing) == 1 else 'them'}")
+    return "; ".join(notes) or None
+
+
 @dataclass
 class SmokeResult:
     chart: str
@@ -209,10 +254,11 @@ def _mixed_queries(chart, spec: DashboardSpec, ds) -> list[dict]:
 
 def smoke_chart(chart, spec: DashboardSpec, resolution: Resolution, client: SupersetClient) -> SmokeResult:
     ds = resolution.for_chart(chart.dataset)
+    queries = [_with_time_range(q, chart, ds) for q in (
+        _mixed_queries(chart, spec, ds) if chart.type == "mixed" else [_query_for(chart, spec)])]
     ctx = {
         "datasource": {"id": ds.id, "type": "table"},
-        "queries": [_with_time_range(q, chart, ds) for q in (
-            _mixed_queries(chart, spec, ds) if chart.type == "mixed" else [_query_for(chart, spec)])],
+        "queries": queries,
         "result_format": "json",
         "result_type": "full",
     }
@@ -226,9 +272,10 @@ def smoke_chart(chart, spec: DashboardSpec, resolution: Resolution, client: Supe
         return SmokeResult(chart.name, False, False, f"unparseable chart/data response: {e}")
     if rows == 0:
         return SmokeResult(chart.name, True, True, "query succeeded but returned 0 rows")
-    fit = _fit_warning(chart, spec, result) or _window_warning(chart, rows)
-    if fit:
-        return SmokeResult(chart.name, True, True, f"{rows} rows; {fit}")
+    notes = [w for w in (_fit_warning(chart, spec, result) or _window_warning(chart, rows),
+                         _empty_values_warning(chart, queries, result, ds)) if w]
+    if notes:
+        return SmokeResult(chart.name, True, True, f"{rows} rows; " + "; ".join(notes))
     return SmokeResult(chart.name, True, False, f"{rows} rows")