Skip to content

Move Otl's three non-shaping tenants out: OtlData, OtlTags and the debugOTL trace (#160) - #198

Merged
jakejackson1 merged 4 commits into
gravitypdffrom
refactor/160-otl-tenants
Sep 17, 2026
Merged

jakejackson1 merged 4 commits into
gravitypdffrom
refactor/160-otl-tenants

Conversation

@jakejackson1

@jakejackson1 jakejackson1 commented Sep 16, 2026

Copy link
Copy Markdown
Member

Part of #160: the sixth Order checkbox, "Otl's three non-shaping tenants (5)". This moves all three out of Otl. Once it merges, the checkbox can be ticked.

It's one PR with one commit per tenant, and each commit can be reviewed on its own. They share no code, and all three are pure refactors: no fixture moves.

Base. refactor/160-otldump-gsub-array (#190). #177 and #178 were squash-merged into gravitypdf, and the stack below this PR was rebased onto it. This branch was then rebased with --onto from the old #190 head (d820375) to the new one (e5db3e5). #186, #188 and #190 are still open, so the base stays #190.

1. Script and language tags → Shaper\OtlTags

_getOTLscriptTag and _getOTLLangTag were private lookup tables. They're now OtlTags::script() and OtlTags::language(), and Otl::selectScriptAndLanguage() is their only caller.

  • The Indic switch is now a v2 tag → original tag table.
  • The useOTL gate is now a method that returns the group's bit.
  • script() drops $mode, which the old method never read.

Exact results. Before deleting the old private methods, I compared them with the new ones through reflection:

  • Script: 107,016,000 cases. Every Ucdn script block with its own tag and five others × 4 shapers × 8 useOTL masks × 3,000+ offered-script sets.
  • Language: 517,720 cases. Every Ucdn language, alone and with region and script subtags, against every OpenType language tag.
  • None differed.

Two existing defects are kept exactly and filed separately, not fixed here:

2. Debug trace → OtlDump::shapingStep()

_dumpproc isn't dead code. Otl::$debugOTL is a public var, off by default, and $mpdf->otl is public, so a caller can switch the trace on. Nothing in src/, tests/ or utils/ does. The trace ends the run with exit.

It's now a static on OtlDump that returns the HTML, placed beside the other report* methods. The 28 call sites echo OtlDump::shapingStep($this->OTLdata, …).

  • Being static, Otl never builds a dump.
  • OtlDump (and TTFontFile under it) only loads once the trace is on.
  • Nothing about the shaper's normal path changes.

Exact output. Five outputs were captured from _dumpproc before it was deleted, and the four unit tests expect four of them. The whole trace of a two-Beh run is byte-identical before and after. Because of the exit, the end-to-end test (ShapingTraceTest::testTheShaperTracesTheRunFromBeginningToEnd) runs the shaper in a child process (tests/Mpdf/Fixtures/shaping-trace.php).

3. OTLdata buffer operations → Shaper\OtlData

old Otl method new
splitOTLdata OtlData::split
sliceOTLdata OtlData::slice
prependOTLchar OtlData::prependChar
removeChar OtlData::removeChar(…, $encoding)
replaceSpace OtlData::nbspToSpace(…, $encoding)
trimOTLdata OtlData::trim

All callers in src/ now call OtlData directly: Mpdf (28 calls), Bidi (9), Tag\BlockTag and Tag\Br.

Why static functions over the same arrays, not a value object. I checked StateSnapshot first, and it settles the question:

  • Runs are copied by assignment throughout Mpdf ($savedPreOTLdata, $objattr['OTLdata'], textbuffer[..][18]), and each copy is expected to be independent.
  • textbuffer and OTLdata are exactly what Mpdf::getStateSnapshot() captures for the table-of-contents look-ahead. StateSnapshot copies arrays by value but shares any object inside them.
  • So a run object trimmed or split during a look-ahead would stay changed after restoreStateSnapshot().
  • A real object would need a clone at every one of those copies, across hundreds of lines of Mpdf. That's a different and riskier change than this checkbox.

Encoding. removeChar and replaceSpace read Mpdf::$mb_enc through Otl's $mpdf, so the static versions take the encoding as an argument.

  • Bidi::prepare() took an Otl only for removeChar. It now takes the encoding instead: Bidi is fork-only, and Mpdf is its only caller.
  • Mpdf no longer creates an Otl just before calling it. ServiceFactory always sets $otl, so that guard never fired.

BC. All six stay on Otl as one-line @deprecated delegates, the way #167 kept unicode_hex(). Otl is reachable through the public Mpdf::$otl, and upstream mpdf/mpdf exposes splitOTLdata, sliceOTLdata, removeChar, replaceSpace and trimOTLdata as public. prependOTLchar is fork-only (#145) but was public here too.

Exact results. Before the old methods were replaced, 200,000 random runs and operations were compared between old and new. The inputs covered empty, null and false runs, missing parts, out-of-order GPOSinfo, four-byte characters, both encodings, and cut points at 0, at the end and past the end. Return values, by-reference changes, warnings and exceptions were all identical.

Tests

  • tests/Mpdf/Shaper/OtlDataTest.php (25 tests, all on hand-built OTLdata arrays): cut at 0, at the end and between cut and restart; restart '' and 0; GPOSinfo ordering; removal at either end, and positions counted in characters with a four-byte emoji ahead; a no-break space the run no longer holds as U+00A0; U+3000 trimming; one-sided trims; all spaces; null and char_data => null; slice/prependChar of []. A group cut down to nothing is compared as a string, because substr() past the end returns false on PHP 5.6.
  • tests/Mpdf/Shaper/OtlTagsTest.php (32 tests): each useOTL group including both CJK ends and Hangul; v2 vs original Indic tags, with and without a shaper; the An Indic run the font has no tag for is laid out with another Indic script's original tag #192 fall-through; DFLTdfltlatn; language with region, script, script+region, upper case, unknown language, missing DFLT, empty tag, empty offer.
  • tests/Mpdf/ShapingTraceTest.php (5 tests), described above.
  • OtlTest: the six deprecated delegates return what OtlData returns.
  • BidiTest: Bidi::prepare() removes RLE/PDF from the text and its run, and sets their levels.

Mutation check

47 mutants: 24 in OtlData, 16 in OtlTags, 5 in shapingStep, 1 in Otl's trace wiring and 1 in a delegate. Each was run against the full suite, snapshots included.

Caught by a unit test only:

  • split: restart 0 no longer read as none; first part cut at the restart; no ksort of either part.
  • slice: <<= at the end; no ksort.
  • prependChar: no defaults for []; GPOSinfo not shifted.
  • removeChar: GPOSinfo of the removed character kept; >>=.
  • nbspToSpace: uni set without checking it's 160.
  • trim: U+3000 ignored at either end; >=> for either end.
  • OtlTags: every group bit and range end; original tags tried without a shaper; DFLT/dflt order; language not lower-cased; the DFLT-only fallback made unconditional.
  • shapingStep: all 5 mutants, plus the Otl wiring and the delegate mutants.

Also caught by snapshots and golden masters:

mutant also caught by
split >=> at the restart AutoFontSnapshotTest
removeChar counting bytes / ignoring its encoding DictionaryLineBreakingSnapshotTest, AutoFontSnapshotTest, BidiTest
nbspToSpace counting bytes RtlSnapshotTest, IndexOfFiftyEntriesSnapshotTest, LongTableOfContentsSnapshotTest
trim left/right flags swapped RtlSnapshotTest, AutoFontSnapshotTest, ZeroFontSizeSnapshotTest
trim without its is_array guard Issue905Test, JustifySnapshotTest, RtlSnapshotTest
useOTL gate removed 11 shaping golden masters
latn fallback removed TaiThamSakotSnapshotTest, SeaTest, Carlito golden master
#192 fixed NotoSansGurmukhiUI shaping golden master
whole-list fall-through; original tag reported as v2; no DFLT language fallback AutoFontSnapshotTest

Survivors, all equivalent or unreachable:

/simplify follow-up

The last commit applies what four review agents found:

  • OtlTags::script() now walks the tag table once instead of calling array_search and then array_slice. Over 53,508,000 cases it gives the same result as the committed version.
  • BidiTest::sort() now reuses chunk(), and its docblock is back in the right place.
  • Three comments that restated code or said something wrong were dropped.

Suggestions not taken:

  • A private Otl wrapper around the 28 trace call sites. It's optional, and the call sites were already this shape.
  • A fixed temp dir for the trace subprocess. The font cache is keyed by family name and only rewritten when the file size changes, so a stale cache could survive between runs.
  • Dropping Tag::$otl. Tag\Columns still passes it on, and removing it changes the constructor signature.

Verification (after the rebase onto e5db3e5, re-run after the /simplify commit)

  • Full suite: 2053 tests, 5904 assertions, 1 skipped. That's 64 new tests on top of the base.
  • Snapshots: 79 tests, 110 assertions. Imagick and Ghostscript are present, and no byte-difference notes were printed. Line-breaking, bidi, hyphenation, kashida and WriteHtml tests pass when run on their own too (76 tests).
  • Golden masters regenerated from cold: tests/Mpdf/tmp/mpdf, tmp/mpdf and tmp/ttfontdata deleted, then fontcache:update, otldump:update, shaping:update and subset:update for all fonts. git status -- tests/data stays empty.
  • composer cs is clean.
  • phpstan: 33 errors, the known baseline skew, all in Gif/* and Writer/FontWriter.php. None are in touched files.

Refs #160, #191, #192.

🤖 Generated with Claude Code

jakejackson1 and others added 4 commits September 17, 2026 11:40
…OtlTags (#160)

_getOTLscriptTag and _getOTLLangTag were lookup tables living in the shaper as private methods.
They move to Shaper\OtlTags as script() and language(), with the same results: the Indic switch
becomes a table of v2 tag to original tag, and the useOTL gate a method returning the group's bit.

script() drops the $mode argument, which the old method never read.

Checked against the old private methods before they were deleted: 107,016,000 script cases (every
Ucdn script block, with its own tag and five others, four shapers, eight useOTL masks and 3,000
random offered-script sets) and 517,720 language cases (every Ucdn language, with region and
script subtags, against every OpenType language tag), none different.

Two behaviours the move keeps and does not fix: a v2 Indic tag falls through to the original tags
of the scripts listed after it (#192), and the language-and-region lookup never matches because
Ucdn keys the regions 'zh-hk' (#191).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…160)

_dumpproc was debug reporting living in the shaper. It is reachable, though nothing in src/,
tests/ or utils/ reaches it: Otl::$debugOTL is a public var, off by default, and $mpdf->otl is
public, so a caller may still switch the trace on. It is not dead code, so it moves rather than
goes.

It becomes OtlDump::shapingStep(), returning the HTML its 28 call sites echo. Static, so that Otl
does not construct a dump or depend on one: the class is loaded only once the trace is on, and
the shaper hands over the run it is tracing.

The five outputs the new tests expect were captured from _dumpproc before it was deleted, and the
whole trace of a two-Beh run through the joining fixture is byte-identical before and after.
Because the trace ends the run with exit, that last test runs it in a process of its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…er (#160)

splitOTLdata, sliceOTLdata, prependOTLchar, removeChar, replaceSpace and trimOTLdata were buffer
surgery on OTLdata that Mpdf, Bidi and two tags called through Otl. They move to Shaper\OtlData
as split, slice, prependChar, removeChar, nbspToSpace and trim, static and operating on the same
arrays, and every caller in src/ goes there directly.

They stay functions over arrays rather than becoming a value object. Runs are copied by assignment
all through Mpdf - into $savedPreOTLdata, $objattr['OTLdata'], textbuffer[18] - and each copy is
relied on to be independent. The textbuffer and OTLdata are also what Mpdf::getStateSnapshot()
captures for a look-ahead, and StateSnapshot copies arrays by value but shares any object inside
them, so a run trimmed during the look-ahead would stay trimmed after restoreStateSnapshot().

removeChar and nbspToSpace read Mpdf::$mb_enc through Otl's $mpdf; they now take the encoding.
Bidi::prepare() needed its Otl only for removeChar, so it takes the encoding instead, and Mpdf no
longer has to make sure an Otl exists before calling it.

The six stay on Otl as deprecated one-line delegates: Otl is reachable as the public Mpdf::$otl,
and upstream has all of them public but prependOTLchar.

Checked against the old methods before they were replaced: 200,000 random runs and operations -
empty, null and false runs, missing parts, out-of-order GPOSinfo, four-byte characters, both
encodings - with identical results, by-reference changes, warnings and exceptions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hunk builder (#160)

Cleanups from review of the three moves:
- OtlTags::script() walks its table once from the text's tag, rather than slicing it
  from an array_search over its keys. Checked against the committed version over
  53,508,000 cases, none different.
- BidiTest::sort() builds its chunk through chunk(), and gets back the docblock the new
  prepare() test had been put under.
- Drop a comment that restated nbspToSpace's name, a sentence from shapingStep's docblock
  that did not explain anything, and a claim in OtlDataTest that its runs are grouped as
  getBasicOTLdata() groups them, which leaves out marks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jakejackson1
jakejackson1 force-pushed the refactor/160-otl-tenants branch from 37f1e92 to a398165 Compare September 17, 2026 01:46
@jakejackson1
jakejackson1 changed the base branch from refactor/160-otldump-gsub-array to gravitypdf September 17, 2026 01:47
@jakejackson1

Copy link
Copy Markdown
Member Author

Rebased: #190 was squash-merged into gravitypdf (f9b3c0e), so this stack still carried its pre-squash commits. Retargeted to gravitypdf and rebased with --onto gravitypdf e5db3e5. Only this PR's own four commits remain; no conflicts. Suite, snapshots and cs pass on the new head, with no fixture movement.

@jakejackson1 jakejackson1 reopened this Sep 17, 2026
@jakejackson1
jakejackson1 merged commit 58b6ed7 into gravitypdf Sep 17, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

create-upstream-pr enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant