Skip to content

Match each class-based chained rule against its own backtrack and lookahead positions (#170) - #195

Merged
jakejackson1 merged 2 commits into
gravitypdffrom
fix/170-class-rule-positions
Sep 17, 2026
Merged

jakejackson1 merged 2 commits into
gravitypdffrom
fix/170-class-rule-positions

Conversation

@jakejackson1

@jakejackson1 jakejackson1 commented Sep 16, 2026

Copy link
Copy Markdown
Member

Closes #170

What

TTFontFile::_getGSUBarray() built a class-based chained context rule's (GSUB Type 6 Format 2) backtrack and lookahead sequences into $backtrackGlyphs and $lookaheadGlyphs, and only emptied them when a rule named no positions at all. A rule with fewer positions than the rule read before it kept that rule's extra ones. It got a matchback and nBacktrack/nLookahead that were too long. For an Arabic joining form (isol/fina/fin2/fin3/medi/med2/init, script arab), the extra positions also went into the prel/postl that Shaper\Arabic requires the text to hold. So the form substitution only fired where the text happened to match the earlier rule's context as well.

Each rule now gets both sequences from a new classSequences() helper, which builds them fresh the way coverageSequences() already does for Format 3 and Type 8, and the way OtlDump already read them:

-	if (!$rule['BacktrackGlyphCount'] || !$this->keepsEarlierRulePositions()) {
-		$backtrackGlyphs = [];
-	}
-	for ($gcl = 0; $gcl < $rule['BacktrackGlyphCount']; $gcl++) { ... }
-	(and the same for the lookahead)
+	list($backtrackGlyphs, $lookaheadGlyphs) = $this->classSequences($subtable, $rule);

After that, keepsEarlierRulePositions() has nothing left to decide. Nothing else called it, so the hook #190 added and OtlDump's override are both deleted.

The comment above the two variables' initialisation now only mentions #189. Type 5 Formats 1 and 2 still read them without setting them, and the stacked PR for #189 fixes that.

Test font and shaping test

No font in the corpus reaches this (see below), so NotoSansArabic-GSUB62Positions-Synthetic.ttf makes it visible. It is NotoSansArabic-Joining-Subset.ttf (Noto Sans Arabic 2.012, OFL 1.1), changed in fontTools 4.59.2 in two ways. Its GSUB is replaced by one fina Type 6 Format 2 lookup (for DFLT and arab) plus the Single Substitution it nests, and name IDs 1, 4 and 6 are renamed.

  • Input class 1 is alef. Its rule has two backtrack positions (beh, then alef) and two lookahead positions (low alef, low alef), and gives alef its final form.
  • Input class 2 is beh, and its rule is read after alef's. It has one backtrack position (beh) and one lookahead position (low alef), and gives beh the dotless final form (uni066E.fina).

U+08AD LOW ALEF doesn't join, so a beh before it is final.

ChainedClassRulePositionsTest records the drawn glyphs with TextRecordingMpdf. The output is in visual order, and PUA E000 = uni0627.fina, E002 = uni066E.fina:

Run (logical) Before After hb-shape 14.3.1
alef beh alef lowalef lowalef (alef's rule) 08AD 08AD E000 0628 0627 same uni08AD uni08AD uni0627.fina uni0628 uni0627
beh beh lowalef lowalef (the carried 2nd backtrack position fails) 08AD 08AD 0628 0628 08AD 08AD E002 0628 uni08AD uni08AD uni066E.fina uni0628
alef beh beh lowalef (the carried 2nd lookahead position fails) 08AD 0628 0628 0627 08AD E002 0628 0627 uni08AD uni066E.fina uni0628 uni0627

HarfBuzz agrees with mPDF after the fix on all three runs. The new font's fontcache fixture has the fixed table: rtlSUB00628prel {"1": ["00628"]}, postl {"1": ["008AD"]}.

GsubArrayTest now expects the fixed behaviour. testTheParserKeepsAnEarlierClassRulesExtraPositions is renamed to testTheParserReadsEachClassRuleWithItsOwnPositions and expects nBacktrack [2, 1] and a one-position matchback. The #189 test keeps describing that bug, but the chained rule it inherits from now has one backtrack position instead of two, so its expected prel is ['00043'].

Fixtures

Regenerated from cold: I deleted tests/Mpdf/tmp/mpdf, tmp/mpdf and tmp/ttfontdata, then ran fontcache:update, otldump:update, shaping:update and subset:update all. git status -- tests/data showed only the new font's five files (ttf, fontcache, otldump, shaping, subset). No existing fixture moved. That matches the issue's measurement: the corpus has no 6.2 rule followed by a shorter one that reaches the output.

The new font's shaping master has no visible form: the golden master's Arabic run uses letters this subset doesn't have. The shaper is covered by ChainedClassRulePositionsTest instead.

Mutation check

With keepsEarlierRulePositions() put back (returning true, with the dump's false override) and caches cleared, 5 tests fail:

  • ChainedClassRulePositionsTest data sets "the second rule, where the first rule's second backtrack position does not hold" and "... second lookahead position does not hold"
  • GsubArrayTest::testTheParserReadsEachClassRuleWithItsOwnPositions
  • GsubArrayTest::testAPlainContextRuleOfAnArabicFormCarriesTheLastChainedRulesSequences (the carried prel goes back to two positions)
  • ParserGoldenMasterTest for NotoSansArabic-GSUB62Positions-Synthetic

Mirroring upstream

Upstream has no hook. In mpdf/mpdf development (744f75a), TTFontFile::_getGSUBarray()'s LookupType 6 Format 2 loop (foreach ($Lookup[$i]['Subtable'][$c]['ChainSubClassSet'] as $inputClass => $cscs), around lines 2780–2810) has:

if ($rule['BacktrackGlyphCount']) {
	for ($gcl = 0; $gcl < $rule['BacktrackGlyphCount']; $gcl++) {
		...
		$backtrackGlyphs[$gcl] = ...;
	}
} else {
	$backtrackGlyphs = [];
}

and the same if/else for $lookaheadGlyphs. The mirror sets $backtrackGlyphs = []; and $lookaheadGlyphs = []; before each for, and drops the else branches. Upstream's OtlDump::_getGSUBarray() still has its own copy of the loop with the same if/else (around line 2481), so the mirror should reset there too. The fork fixed its copy in 271b425. The upstream PR also needs a CHANGELOG.md entry. The test font and ChainedClassRulePositionsTest can be taken over as they are.

Base

Stacked on #190 (refactor/160-otldump-gsub-array) → #188#186gravitypdf. #177 and #178 were squash-merged into gravitypdf. The stack was then rebased onto gravitypdf, and #190's head moved from d820375 to e5db3e5. I rebased this branch with git rebase --onto refactor/160-otldump-gsub-array d820375 and force-pushed it with a lease. There were no conflicts. #186, #188 and #190 are still open, so the base stays refactor/160-otldump-gsub-array.

/simplify

Applied:

Skipped:

  • extracting the drawn() helper, which this test shares with 7 existing shaping tests, into a trait. That would mean editing those tests, which is outside this diff.
  • hoisting OtlDump::classNames() into TTFontFile, which is outside the files this change touches.
  • building one Mpdf for all three data sets, since each sibling test builds one per run

Verification

  • vendor/bin/phpunit (after the rebase): 1999 tests, 5781 assertions, 1 skipped. The base e5db3e5 has 1989 / 5761 / 1. The 10 extra tests are the new font's golden-master data sets plus the three shaping runs.
  • Golden masters regenerated from cold again after the rebase: git status -- tests/data is clean, so nothing moved and the new font's committed fixtures reproduce exactly.
  • vendor/bin/phpunit --group=snapshot: OK, 79 tests, 110 assertions.
  • composer cs: clean.
  • phpstan: 33 errors, the same as the baseline.

🤖 Generated with Claude Code

jakejackson1 and others added 2 commits September 17, 2026 11:40
…kahead positions (#170)

The parser built a Type 6 Format 2 rule's backtrack and lookahead sequences into arrays it
only emptied when a rule named no positions at all. A rule naming fewer positions than the
rule read before it kept that rule's extra ones: a matchback and counts too long, and, for
an Arabic joining form, a prel and postl the shaper then required of the text. The form
substitution only fired where the text happened to hold the earlier rule's context as well.

Both arrays are now built fresh for every rule, which is what the dump already did. That
leaves nothing for keepsEarlierRulePositions() to decide, so the hook and the dump's
override of it go.

No font in tests/data/ttf reaches this, and regenerating every golden master from cold moves
none of them. NotoSansArabic-GSUB62Positions-Synthetic makes it visible: it is
NotoSansArabic-Joining-Subset (Noto Sans Arabic 2.012, OFL 1.1) with its GSUB replaced, in
fontTools 4.59.2, by one 'fina' Type 6 Format 2 lookup and the Single Substitution it nests,
and name IDs 1, 4 and 6 renamed. Alef's rule gives it the final form after beh, alef and
before two low alefs; beh's rule, read after it, gives beh the dotless final form after one
beh and before one low alef. Before, beh's rule asked for alef two back and a second low
alef, so beh beh low-alef low-alef and alef beh beh low-alef both drew a plain beh.
hb-shape 14.3.1 draws the dotless final form in both, as mPDF now does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ce (#170)

classSequences() returns both sequences fresh for each rule, as coverageSequences() does for
a Coverage-based one, so a rule cannot pick up positions from the one before by how the
walk is written rather than by a reset at the top of the loop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jakejackson1
jakejackson1 force-pushed the fix/170-class-rule-positions branch from 7168e2e to 46622c6 Compare September 17, 2026 01:46
@jakejackson1
jakejackson1 changed the base branch from refactor/160-otldump-gsub-array to gravitypdf September 17, 2026 01:47
@jakejackson1

Copy link
Copy Markdown
Member Author

Rebased: #190 was squash-merged into gravitypdf (f9b3c0e), so this stack still carried its pre-squash commits. Retargeted to gravitypdf and rebased with --onto gravitypdf e5db3e5. Only this PR's own commits remain; no conflicts. Suite, snapshots and cs pass on the new head, with no fixture movement.

@jakejackson1 jakejackson1 reopened this Sep 17, 2026
@jakejackson1
jakejackson1 merged commit d48f304 into gravitypdf Sep 17, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working create-upstream-pr

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The parser keeps a class-based chained rule's backtrack and lookahead positions into the next rule

1 participant