Skip to content

chore(benchmarks): time the preview pipeline, stage by stage - #499

Merged
DemchaAV merged 1 commit into
developfrom
chore/preview-cost-probe
Aug 4, 2026
Merged

chore(benchmarks): time the preview pipeline, stage by stage#499
DemchaAV merged 1 commit into
developfrom
chore/preview-cost-probe

Conversation

@DemchaAV

@DemchaAV DemchaAV commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Why

DocumentSession.toImages(dpi) reaches a raster through a chain of stages — open the
session, compose, layout, build a PDDocument and save() it to bytes, Loader.loadPDF,
PDFRenderer — and the cost of each was assumed rather than measured. A probe that
prints them is the cheapest way to stop guessing.

An earlier revision of this branch drew a conclusion from those numbers: that a direct
Java2D backend would delete three stages and leave rasterisation untouched, bounding the
win at a few milliseconds. That was wrong on both halves, and the probe now says so in
its own Javadoc rather than in a commit message nobody reads:

  • PDF encode is not serialisation. Before save() the backend creates pages, walks
    the whole LayoutGraph, paints every fragment, resolves links and bookmarks, and
    applies headers, footers, metadata, watermark and protection. A direct renderer
    replaces that painting work; it does not delete it.
  • PDFRenderer is not pixel production alone. It parses and interprets PDF operators
    and then paints through Java2D. A direct renderer skips the interpretation and keeps
    the painting.

A direct backend therefore changes work in both stages, so no measurement of the present
pipeline can predict the result. Comparing the two designs needs a second implementation
to measure against — which is exactly what this probe does not substitute for.

What

One new file, benchmarks/.../PreviewCostProbe.java. It reports each stage warm, as a
median with min/max, on two canonical workloads (a one-page CV and a three-page proposal).

Measurement decisions worth naming:

  • Every DPI renders from a freshly parsed document and a fresh PDFRenderer, so a later
    DPI does not inherit the glyph and resource caches an earlier one warmed. The parse
    sits outside the timed region.
  • toImages is measured on its own sessions rather than after four raster passes over
    the same document.
  • Opening the session is its own row. The constructor resolves the backend's measurement
    services before a single node is composed, and folding that into the compose figure
    attributed engine setup to the DSL — on the CV workload it was roughly a fifth of what
    the probe called "compose".
  • Totals are summed within an iteration, so the reported median is a median of real
    totals rather than a sum of independent medians.

Tests

./mvnw -B -ntp clean verifyBUILD SUCCESS. The probe itself was run; both workloads
report, and the sum of the stages row tracks the measured toImages figure it is meant
to account for (CV: 17.29 ms sum vs 17.04 ms fresh-session toImages).

This is a probe, not a gate: it is a main in benchmarks/, run on demand, and nothing
in CI depends on its numbers.

@DemchaAV
DemchaAV force-pushed the chore/preview-cost-probe branch from 9a3962b to 97b8c7e Compare August 4, 2026 07:45
@DemchaAV

DemchaAV commented Aug 4, 2026

Copy link
Copy Markdown
Owner Author

Reworked as a neutral pipeline probe, and rebased onto develop (39de48ab). Head is now 97b8c7eb.

Your blocker was right, and it was mine to make: the probe measured the pipeline correctly and then interpreted it as a bound on a design that does not exist yet. Both stages I called fixed are nothing of the sort — toPdfBytes() walks the whole LayoutGraph and paints every fragment before save() ever runs, and PDFRenderer interprets PDF operators before it paints. A direct renderer changes work in both. The "3 ms" figure was an artefact of that framing.

Removed: removable / not removable, the saving estimate, and the design conclusion — from the javadoc, the printed labels and the commit message. The stages now read as what they are: PDF encode: paint graph + save, PDF parse: Loader.loadPDF, PDFRenderer <dpi>, all pages. The javadoc states in its own section that these numbers do not predict a different pipeline and that comparing designs needs a second implementation.

The three measurement fixes, and what they cost:

  • Fresh parse and fresh PDFRenderer per DPI. This one changed the answer. Sharing a renderer handed each later DPI the caches the first warmed, and the old numbers understated rasterization by 13% on the CV (9.93 → 11.26 ms) and 28% on the proposal (46.93 → 59.88 ms). The parse is outside the timed region.
  • toImages on its own sessions, not after four raster passes — reported twice, layout-cached and from a fresh session.
  • Totals summed inside an iteration, then medianed, with a note in the javadoc saying why stage medians cannot be added.

Current output:

=== preview pipeline, stage by stage (canonical CV, ModernProfessional) ===
pages=1 pdfBytes=2529
compose (DSL build)               0.53 ms
layout (compile graph)            2.11 ms
PDF encode: paint graph + save    2.63 ms
PDF parse: Loader.loadPDF         0.59 ms
PDFRenderer 72dpi, all pages      9.17 ms
PDFRenderer 96dpi, all pages     11.26 ms
PDFRenderer 150dpi, all pages    17.22 ms
PDFRenderer 96dpi, first page    11.14 ms
sum of the stages at 96dpi       17.43 ms
toImages(96), layout cached      14.49 ms
toImages(96), fresh session      16.13 ms

The gap between the summed stages and toImages is itself informative and now visible: the probe parses separately per stage and shares nothing, while toImages parses once — which is a property of the measurement, not of the pipeline.

./mvnw -B -ntp clean verify on the rebased branch — BUILD SUCCESS, 692 tests in the closing module. Probe run end to end, exit 0.

DocumentSession.toImages(dpi) reaches a raster through compose, layout,
building a PDDocument and saving it to bytes, Loader.loadPDF, and
PDFRenderer. The probe times each of those separately, warm, on two
canonical workloads, so the shape of the cost is measured rather than
assumed.

It deliberately draws no conclusion about a direct Java2D backend. The
encode stage is not serialisation — before save() the backend creates pages,
walks the whole LayoutGraph, paints every fragment, resolves links and
bookmarks and applies the page chrome. The PDFRenderer stage is not pixel
production alone — it interprets PDF operators and then paints them through
Java2D. A direct renderer would change work in both stages, so no stage here
is removable, and no measurement of this pipeline predicts the cost of a
different one. That comparison needs a second implementation.

Three measurement details the numbers depend on:

- every DPI renders from a freshly parsed document and its own PDFRenderer,
  because sharing one hands each later DPI the caches the first one warmed —
  measured at 13% on the CV and 28% on the proposal;
- toImages runs on its own sessions rather than after four raster passes
  over the same document, in both the layout-cached and from-scratch case;
- totals are summed inside an iteration and the median taken of those, since
  medians of separate stages do not add to the median of their sum.

It joins the probe family already in this module — AllocationRateProbe,
AutoSizeMeasureProbe, ChartAllocProbe, FontEmbedProbe — reuses
CanonicalBenchmarkSupport, reads nothing private and changes no src/main
code. benchmarks is a development module and ships to no registry.

Run end to end, exit 0.
@DemchaAV
DemchaAV force-pushed the chore/preview-cost-probe branch from 97b8c7e to b49b8ed Compare August 4, 2026 12:08
@DemchaAV DemchaAV changed the title chore(benchmarks): measure what a direct preview backend would actually save chore(benchmarks): time the preview pipeline, stage by stage Aug 4, 2026
@DemchaAV
DemchaAV merged commit 17d393d into develop Aug 4, 2026
10 checks passed
@DemchaAV
DemchaAV deleted the chore/preview-cost-probe branch August 4, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant