Skip to content

perf(export): hand large copies to the isolate in chunks (#1537) - #1659

Merged
ZhuchkaTriplesix merged 1 commit into
devfrom
issue/1537-export-handoff
Oct 17, 2026
Merged

ZhuchkaTriplesix merged 1 commit into
devfrom
issue/1537-export-handoff

Conversation

@ZhuchkaTriplesix

Copy link
Copy Markdown
Member

Closes #1537. The dead resultGridAsCsvAsync was already removed in #1606; this is the other half, the hand-off stall.

What stalls

DataExportService.formatAsync ran one Isolate.run whose closure captured all rows. The captured object graph is copied on the calling (UI) isolate before the worker starts, so copying 50,000 × 20 cells to the clipboard stopped the UI for ~20–30 ms.

Change

  • Up to handOffCells (20,000 cells) nothing changes: one isolate, the whole table.
  • Above that, the rows go over in chunks of about 20,000 cells. Each chunk is formatted on its own isolate, the event loop runs between chunks, and the parts are joined.
  • formatChunk makes each part so that the join is exactly what formatCsv / formatJson / formatMarkdownTable / formatSqlInsertDump give for all rows:
    • CSV and Markdown: the header goes in the first chunk only;
    • JSON: [ in the first chunk, ] in the last, and the items are cut out of the indented array;
    • SQL dump: rows are independent.
  • The isolate calls are separate functions so that each closure captures only its own arguments. A closure created inside formatAsync would carry the whole rows along; in my first measurement that made chunking slower than no chunking.
  • formatMarkdownTable is split into header and rows helpers. Its output is unchanged.

Measured

Standalone Dart AOT, the issue's shape: 50,000 × 20 CSV, Timer.periodic(1 ms) on the calling isolate, longest gap during the call, 3 runs.

longest UI gap total
one closure (before) 20–28 ms 77–96 ms
1,000-row chunks 2–4 ms 50–58 ms
2,000–10,000-row chunks 3–16 ms 55–74 ms

20,000 cells per chunk is the 1,000-row case at 20 columns. The clipboard call itself (Clipboard.setData with one big string) is not part of this.

Tests (test/shared/data_export_service_test.dart)

For every format, chunks of 1, 3, 7, 22 and 23 rows joined equal the whole document. The rows include quotes, commas, newlines, |, NULL and short rows. formatAsync on ~40,000 cells (3 chunks) equals the whole document.

Not run with flutter test locally; CI decides.

formatAsync ran one Isolate.run whose closure captured all rows. The
captured object graph is copied on the calling isolate before the worker
starts, so copying 50,000 x 20 cells to the clipboard stopped the UI for
about 20-30 ms.

Above about 20,000 cells (handOffCells) the rows now go over in chunks
of that size, each formatted on its own isolate, with the event loop
running between them; the parts are joined. formatChunk produces each
part so that the join is exactly the format* output: the header (and
JSON's '[') in the first chunk, ']' in the last, the JSON items split
out of the indented array. The isolate calls live in separate functions
so a closure captures only its chunk, not formatAsync's whole rows.

Standalone Dart AOT, 50,000 x 20, Timer.periodic(1 ms) on the caller:
longest gap 21 ms with one closure, 2-4 ms with 1,000-row chunks; total
time ~85 ms -> ~55 ms.
@github-actions github-actions Bot added enhancement New feature or request performance Theme parser epic label: performance core Core library logic and services data-grid Interactive data grid, cell editor, filtering, groupings P2 Medium priority / Parity & Refactoring labels Oct 17, 2026
@github-actions github-actions Bot added this to the 0.5.0 milestone Oct 17, 2026
@ZhuchkaTriplesix
ZhuchkaTriplesix merged commit 6d2c6f6 into dev Oct 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core library logic and services data-grid Interactive data grid, cell editor, filtering, groupings enhancement New feature or request P2 Medium priority / Parity & Refactoring performance Theme parser epic label: performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant