Fall back from a v2 Indic tag to its own original tag only (#192)

[jakejackson1]

Closes #192

What

OtlTags::script() walked $originalIndicTags from the run's own v2 tag to the end of the table. So a v2 tag the font doesn't offer reached the original tag of every script listed after it before DFLT/dflt/latn. Bengali text in a font offering deva was laid out with the Devanagari lookups and the original specification's reordering. Devanagari never reached beng, because the result depended on table order. The walk is now one lookup of the run's own original tag:

-		if ($shaper) {
-			$reached = false;
-			foreach (self::$originalIndicTags as $v2 => $original) {
-				$reached = $reached || $v2 === $scripttag;
-				if ($reached && isset($ScriptLang[$original])) {
-					return [$original, true];
-				}
-			}
+		$original = isset(self::$originalIndicTags[$scripttag]) ? self::$originalIndicTags[$scripttag] : '';
+		if ($shaper && $original && isset($ScriptLang[$original])) {
+			return [$original, true];
 		}

The table's docblock no longer says order is load-bearing, because it isn't any more.

HarfBuzz

HarfBuzz 14.3.1 agrees with the issue. hb_ot_all_tags_from_script() (src/hb-ot-tag.cc) lists only the script's own tags, newest first:

  hb_tag_t new_tag = hb_ot_new_tag_from_script (script);
  if (unlikely (new_tag != HB_OT_TAG_DEFAULT_SCRIPT))
  {
    /* HB_SCRIPT_MYANMAR maps to 'mym2', but there is no 'mym3'. */
    if (new_tag != HB_TAG('m','y','m','2'))
      tags[i++] = new_tag | '3';
    if (*count > i)
      tags[i++] = new_tag;
  }

  if (*count > i)
  {
    hb_tag_t old_tag = hb_ot_old_tag_from_script (script);
    if (old_tag != HB_OT_TAG_DEFAULT_SCRIPT)
      tags[i++] = old_tag;
  }

hb_ot_layout_table_select_script() (src/hb-ot-layout.cc) tries those tags, then the defaults:

  for (i = 0; i < script_count; i++)
  {
    if (g.find_script_index (script_tags[i], script_index))
    { ... return true; }
  }

  /* try finding 'DFLT' */
  ...
  /* try with 'dflt'; MS site has had typos and many fonts use it now :( */
  ...
  /* try with 'latn'; some old fonts put their features there even though
     they're really trying to support Thai, for example :( */

So Bengali asks for bng3, bng2, beng, then DFLT, dflt, latn, and never another script's tag. mPDF has no *3 (USE) tags, and adding them would widen this PR, so it stays out.

Test font and shaping test

I checked every font in tests/data/ttf and packages/*/fonts. Several offer another Indic script's original tag without the run's own: Lohit-Kannada and Pothana2000 (deva), FreeSansBold and FreeSerifBold (deva, guru), the Myanmar bundle (mymr), and NotoSansGurmukhiUI-GPOS71-Subset. None of them has glyphs for the script that would fall through, so no page in the corpus can show the bug.

NotoSansBengali-DevaScript-Synthetic.ttf makes it visible. It's Noto Sans Bengali 3.011 (OFL 1.1), subset in fontTools 4.59.2 to space, U+25CC, KA, KHA, GA, SSA, I and VIRAMA, with GPOS dropped and name IDs 1, 4 and 6 renamed. Its GSUB is replaced by two scripts and no Bengali one:

• DFLT: locl gives KA (U+0995) the GA glyph (U+0997)
• deva: locl gives KA the KHA glyph (U+0996)

IndicScriptFallbackTest records what's drawn for KA through TextRecordingMpdf:

	mPDF before	mPDF after	hb-shape 14.3.1
U+0995	0996 (deva)	0997 (DFLT)	uni0997
U+0995 with --script=deva forced			uni0996

OtlTagsTest's pinned case is replaced by a data provider with the issue's four rows, all now DFLT (or '' where the font has no default), plus a font offering both beng and deva, where Bengali still takes beng.

Fixtures

Regenerated from cold. I deleted tests/Mpdf/tmp/mpdf, tmp/mpdf and tmp/ttfontdata, then ran composer fontcache:update all, otldump:update all, shaping:update all and subset:update all. Besides the new font's four fixtures, one existing fixture moved: tests/data/shaping/NotoSansGurmukhiUI-GPOS71-Subset.txt, in its devanagari and bengali runs.

That font offers only gur2 and guru. dev2 and bng2 used to fall through to guru, and the runs were reordered to the original specification (0915 094D 0937 093F → 0915 094D 093F 0937). Now they find neither their own tag nor a default, so no script is chosen and they come back unshaped in logical order. That is what happens to any run in a font with no entry for its script. The font has no glyph for either run, so every glyph is .notdef either way.

HarfBuzz still reorders a run like that: with no script it runs the Indic shaper, to the original specification. mPDF's earlier output matched HarfBuzz here only because guru happened to be picked. The reverse also holds: under DFLT, HarfBuzz uses the default shaper and doesn't reorder, but mPDF does (the new font's bengali run is 0997 09CD 09BF 09B7, and hb-shape gives uni0997 uni09CD uni09B7 uni09BF). Which shaper runs for the chosen tag is a different decision from which tag is chosen, so I opened #199 for it rather than widening this PR.

Mutation check

With the fallthrough walk put back and the caches cleared, 6 tests fail:

• IndicScriptFallbackTest::testABengaliRunIsLaidOutWithTheDefaultScriptRatherThanDevanagari
• OtlTagsTest::testAV2TagFallsBackToItsOwnOriginalTagAndNoOtherScriptsOriginalTag, for "Bengali, where the font offers deva", "Devanagari, where the font offers gujr" and "Tamil, where the font offers only mymr"
• ShapingGoldenMasterTest for NotoSansBengali-DevaScript-Synthetic and NotoSansGurmukhiUI-GPOS71-Subset

Mirroring upstream

Upstream still has the switch in Otl::_getOTLscriptTag() (src/Otl.php:6079 on development, inside if ($shaper) {), with a // fallthrough after each case. For the mirror, each case returns its own original tag or leaves the switch: replace every // fallthrough with break;, for example:

case 'bng2':
	if (isset($ScriptLang['beng'])) {
		return ['beng', true];
	}
	break;

mym2 is last and needs no break. Nothing else changes, and control falls on to the existing DFLT/dflt/latn checks. The upstream PR also needs a CHANGELOG.md entry. The test font, IndicScriptFallbackTest, and the script() rows of OtlTagsTest (as calls to _getOTLscriptTag through reflection) can be carried over. NotoSansGurmukhiUI-GPOS71-Subset isn't upstream, so its fixture move doesn't apply there.

Base

Stacked on #198 (refactor/160-otl-tenants) → #190 → #188 → #186 → gravitypdf. Before pushing I checked all four: all still open, none merged, so this targets refactor/160-otl-tenants at 37f1e92 without a rebase.

/simplify

Applied:

• the original tag is looked up once into $original, instead of twice inside one long isset()
• the OtlTagsTest provider is renamed originalTagFallbacks, because its last row checks that a run's own original tag still wins when another script's tag is also offered

Skipped:

• moving the drawn-codepoints conversion into TextRecordingMpdf. About a dozen shaping tests already repeat that line, so it would mean editing tests this diff doesn't touch.
• trimming the provider to two rows. The four DFLT/'' rows are the issue's table, and together they cover a tag listed both before and after the run's own.

Verification

• vendor/bin/phpunit: 2065 tests, 5924 assertions, 1 skipped. The base was 2053 / 5904 / 1.
• vendor/bin/phpunit --group=snapshot: OK, 79 tests, 110 assertions.
• composer cs: clean.
• phpstan: 33 errors, the same as the base, none in touched files.

🤖 Generated with Claude Code

[jakejackson1]

Rebased: #190 was squash-merged into gravitypdf (f9b3c0e), so this stack still carried its pre-squash commits. This branch is rebased onto its rebased parent with --onto; no conflicts and the base is unchanged. Suite, snapshots and cs pass on the new head, with no fixture movement.

[jakejackson1]

This PR merged into a base branch that had already been squash-merged, so its change never reached gravitypdf. It's carried to gravitypdf by #216.