Skip to content

feat(files): preview .csv and .xlsx as data grids in the docked pane - #1134

Merged
philmerrell merged 3 commits into
developfrom
claude/spreadsheet-file-preview-8135b9
Sep 16, 2026
Merged

philmerrell merged 3 commits into
developfrom
claude/spreadsheet-file-preview-8135b9

Conversation

@philmerrell

Copy link
Copy Markdown
Contributor

Adds spreadsheet and delimited-data previews to the docked pane, alongside the .docx and .pptx viewers from #1122.

.csv is an allowed upload type that had no preview at all — a .md attachment opened a modal and a csv card did nothing, so the only way to look at a data file in the app was to ask the agent to read it back, which is the most expensive way to look at a table.

This does not reverse the earlier .xlsx decision

That decline was of client-side renderers — reproducing Excel's own layout. It still stands: the npm build of SheetJS is frozen on a 2022 release with unfixed advisories, and ExcelJS raises outright on any workbook containing a native chart, which is exactly what create_excel_spreadsheet produces.

A grid of values needs a reader, not a renderer. So:

  • .csv parses in the browser (csv-parse.ts, ~200 lines, no new dependency).
  • .xlsx is read by app-api with openpyxl and sent as rows. The workbook never reaches the browser.

One DataGridComponent draws both.

What reviewers should weigh

New dependency: openpyxl==3.1.5 (MIT, pure Python, one dependency et-xmlfile). It is already what our own create_excel_spreadsheet tool drives inside Code Interpreter, and the rag-ingestion image pulls it via docling — this pin brings it into app-api's own closure, where it was not before. Note upstream is slow-moving: 3.1.5 dates from June 2024 and is still the latest release.

New route: GET /files/{upload_id}/sheet-preview on app-api, cookie-authenticated via get_current_user_from_session like every other user-facing route. It distinguishes 404 / 413 / 415 / 422 because each means something different to the user. The size cap is checked against recorded metadata, so an oversized workbook costs a metadata read rather than a transfer into memory, and the parse runs in asyncio.to_thread because openpyxl is blocking CPU.

The non-obvious part: openpyxl is read twice

data_only=True returns None for any formula Excel never cached — and openpyxl does no evaluation, so every formula in a workbook our own tools wrote reads back as None. A values-only preview would show blanks exactly where the totals belong. The second pass supplies the formula text, so such a cell renders as =SUM(B2:B10) rather than empty. That is both more useful and more honest: the file really does not carry that number yet.

The second pass is lazy — it only runs for a sheet that came back with an empty cell, since that is the only thing a value-less formula can look like. A 50,000-row sheet with no formulas went 0.53s → 0.09s.

Formatting decisions live on the server so the two readers cannot drift: floats lose binary noise (0.1 + 0.2 renders 0.3), a date-formatted cell does not gain the midnight openpyxl invents for it, booleans render TRUE/FALSE, and 007 stays 007. Neither reader does type inference — a preview that prettifies 007 into 7 is answering a different question than the one being asked.

Row caps, measured rather than guessed

The caps are not a limit on what the grid can draw — it virtualises with cdk-virtual-scroll-viewport and recycles row elements, so the DOM is bounded regardless. They bound the server: one JSON response with an openpyxl parse in front of it. On a 20-column sheet:

cap parse JSON gzip
500 0.04s 0.06 MB 0.02 MB
5,000 0.53s 0.68 MB 0.15 MB
50,000 3.80s 7.17 MB 1.51 MB

There is no gzip middleware on app-api, so the JSON column is the wire cost. The cap is 5,000 rows per sheet, under a global cell budget spent against each sheet's own width.

Fetch-on-scroll was considered and rejected on measurement: openpyxl's read-only mode is a streaming parser, so min_row does not seek. Reaching row 45,000 of a 50,000-row sheet costs 2.63s against 2.97s for a complete pass, making paging O(n²) over a scroll — roughly 119s of CPU where one pass costs 3s. For this format, read once and send more.

Two bugs found while verifying in the browser

  • The virtual scroller froze on any second dataset. One viewport serves every dataset the grid is handed — switching worksheet tabs, or previewing a second file without closing the pane. CDK picked up the new length (the spacer grew to full height) but never recomputed the rendered range, so the body stayed pinned to the first screenful while the scrollbar travelled the whole sheet. Measured: at scrollOffset 10000 of a 1,200-row sheet the rendered range was still {start: 0, end: 32}, and checkViewportSize() corrected it to {start: 309, end: 345} on the spot. This affected the .csv path too.
  • Row numbers measured 2.6:1 on the light grid — WCAG AA fail. There is no single neutral step good for both surfaces, so the gutter uses the pair text-gray-500 dark:text-gray-400. Every text pair in the grid now clears AA in both themes (minimum 4.84:1).

Also included

inline-template-backtick-guard.spec.ts — a backtick inside an inline template:/styles: block ends the template literal early; tsc --noEmit passes and only the Angular compiler fails, naming the wrong cause ("Cannot find name 'text'"). It has cost debugging time three times. Now it is a test failure with a message that says what actually happened.

Verification

  • Backend: 8,626 passed, 3 skipped. Includes test_sheet_preview_route.py, which drives route → real service → openpyxl with only S3 and the repository stubbed, and asserts the zip magic PK\x03\x04 never appears in the response body.
  • Frontend: 265 files green on five consecutive full runs (the new spec files reshuffle vitest worker assignment, which is what trips the isolate: false class of flake — develop's 442e574c memo fix holds).
  • Live against dev: uploaded a workbook through the app's own presign/complete endpoints and opened it in the real pane. Formulas render as =SUM(...), the hidden sheet is omitted, 0.1 + 0.2 reads 0.3, and a 1,200-row sheet returns whole in a 311ms round trip and scrolls to row 1,200 with 29–36 rows in the DOM.

Follow-ups (not in this PR)

  • Adding GZipMiddleware to app-api would take that 0.68 MB response to 0.15 MB and help every other endpoint. It needs text/event-stream excluded first — CloudFront already sets compress: false on /api/* for exactly that reason.
  • text-state-warning-600 dark:text-state-warning-400 fails AA in light mode (3.2:1) and appears in ~36 places app-wide. Fixed here in the one instance this PR touches; the sweep is separate.

🤖 Generated with Claude Code

philmerrell and others added 3 commits September 16, 2026 07:48
Adds a third viewer to the preview pane. `.csv` is an allowed upload
type but had no preview at all — a `.md` attachment opens a modal and a
csv card did nothing, so the only way to look at a data file in the app
was to ask the agent to read it back, which is the most expensive way to
look at a table.

Deliberately a data grid, not a spreadsheet view. Cells stay the strings
the file contained: no type inference, no locale-aware numbers, no date
parsing. A preview that renders 007 as 7 is answering a different
question than the one being asked.

This does not reopen the .xlsx decision. That was a decline of the
client-side *renderers* — SheetJS frozen on npm with unfixed advisories,
ExcelJS throwing on any workbook with a native chart. A delimited file
needs a parser, not a renderer, and the parser here is 200 lines with no
new dependency.

- `csv-parse.ts`: RFC 4180 field splitting (quoted delimiters, embedded
  newlines, doubled quotes), delimiter sniffing that scores consistency
  across rows rather than raw frequency, BOM handling, all three line
  endings, and byte/row/column caps.
- `csv-viewer.component.ts`: virtualised with `cdk-virtual-scroll-
  viewport`, so a 40k-row file scrolls rather than showing its first
  screenful. `@angular/cdk` was already a dependency. Virtualising costs
  the `<table>` element, so the grid carries explicit ARIA grid roles and
  reports `aria-rowcount` from the data rather than the rendered window.
- `PREVIEW_KIND_MIMES` becomes a list per kind: the recorded MIME is
  whatever the browser reported at upload, and Windows reports
  `application/vnd.ms-excel` for a .csv whenever Excel is the registered
  handler. The extension still chooses the viewer, so .xls stays out.

Verified in the browser against a 4,001-row file: virtual scroller exact
at depth (row 1001 at scrollTop 32000, 35 rows in the DOM), header and
body sharing one track list, horizontal scroll synced, and every text
pair clearing WCAG AA in both themes (min 4.63:1).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Completes the spreadsheet preview. The `.csv` grid shipped in 2c6aebdf
parses in the browser; a workbook cannot, so app-api reads it and sends
rows. The distinction that unblocked this is that the earlier `.xlsx`
decline was of client-side *renderers* — SheetJS frozen on npm with
unfixed advisories, ExcelJS raising on any workbook with a native chart,
which is what `create_excel_spreadsheet` produces. A grid of values
needs a reader, and openpyxl is already what our own spreadsheet tools
drive inside Code Interpreter.

New dependency: openpyxl==3.1.5 (MIT, pure Python, one dependency).
Already present in the rag-ingestion image via docling; this pin brings
it into app-api's own closure, where it was not before.

Backend
- `sheet_preview.py` reads every visible sheet into display strings,
  streaming with `read_only=True` and stopping at the caps rather than
  walking a whole sheet to discard most of it.
- It reads the workbook TWICE, and that is the point. `data_only=True`
  returns None for any formula Excel never cached — the normal state of
  a workbook openpyxl itself wrote, since openpyxl does not evaluate —
  so a values-only preview would show blanks exactly where the totals
  belong. The second pass supplies the formula text instead.
- Formatting decisions are the server's so the two readers cannot drift:
  floats lose binary noise (0.1+0.2 renders as 0.3), a date-formatted
  cell does not gain the midnight openpyxl invents for it, and a numeric
  string stays a string.
- `GET /files/{id}/sheet-preview`, cookie-authenticated like every other
  user-facing app-api route, distinguishing 404/413/415/422 because each
  means something different to the user. The size cap is checked against
  recorded metadata, so an oversized workbook costs a metadata read
  rather than a transfer into memory.

Frontend
- `DataGridComponent` extracted from the csv viewer: one virtualised
  grid now serves both readers, so the ARIA roles, column sizing, header
  scroll sync and contrast work are shared rather than duplicated.
- `XlsxViewerComponent` takes an upload id rather than bytes, with a tab
  per sheet. `previewFetchesBytes()` keeps the two paths apart, and
  `fetchDocument` now refuses a server-read kind outright — letting one
  through would pull a whole workbook into the browser and discard it.

Also fixes two things found while verifying in the browser: the
truncation notice measured 3.2:1 on the light grid and failed WCAG AA
(-700 clears it at 5.03:1), and the per-character column estimate
clipped symbol-heavy cells like formula text by a few px.

Adds `inline-template-backtick-guard.spec.ts`. A backtick inside an
inline template:/styles: block ends the literal early; tsc passes and
only the Angular compiler fails, naming the wrong cause. It has cost
debugging time three times, twice in this branch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…oller

Two problems, found by questioning why the xlsx preview stopped at 500
rows when the csv one shows 50,000.

**The cap was guessed, not measured.** It is not a limit on what the
grid can draw — the viewer virtualises and recycles row elements, so the
DOM is bounded whatever it is handed. It bounds the server: one JSON
response, and an openpyxl parse in front of it. Measured on a 20-column
sheet:

    cap      parse    JSON      gzip
    500      0.04s    0.06 MB   0.02 MB
    5,000    0.53s    0.68 MB   0.15 MB
    50,000   3.80s    7.17 MB   1.51 MB

There is no gzip middleware on app-api, so the JSON column is the wire
cost. 5,000 buys ten times the rows for half a second.

Fetch-on-scroll was considered and rejected on measurement: openpyxl's
read-only mode is a streaming parser, so `min_row` does not seek.
Reaching row 45,000 of a 50,000-row sheet costs 2.63s against 2.97s for
a complete pass, so paging is O(n²) over a scroll — ~119s of CPU where
one pass costs 3s. For this format, read once and send more.

Two further fixes the measurements exposed:

- The cell budget divided by MAX_COLUMNS_PER_SHEET rather than the
  sheet's own width, charging a 7-column sheet as though it were 64
  columns wide and cutting its rows by an order of magnitude.
- The formula pass is now lazy. It doubles the parse and earns nothing
  on a sheet with no empty cells, since an empty cell is the only thing
  a cached-value-less formula can look like. A 50,000-row sheet with no
  formulas went 0.53s → 0.09s.

**The scroller froze on any second dataset.** One viewport serves every
dataset the grid is handed — switching worksheet tabs, or previewing a
second file without closing the pane. CDK picked up the new length (the
spacer grew to full height) but never recomputed the rendered range, so
the body stayed pinned to the first screenful while the scrollbar moved
over the whole sheet. Measured in the browser: at scrollOffset 10000 of
a 1,200-row sheet the rendered range was still {start: 0, end: 32}, and
checkViewportSize() corrected it to {start: 309, end: 345} on the spot.
The grid now re-measures and rewinds when its rows change — which is the
right behaviour anyway, since a newly chosen sheet should start at its
first row rather than inherit the previous one's offset.

Verified live against dev: the 1,200-row sheet now returns whole in a
311ms round trip and scrolls to row 1,200 with 29-36 rows in the DOM.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@philmerrell
philmerrell merged commit 27c593c into develop Sep 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant