Select ZHH, ZHS and ZHT for a Chinese region in the lang attribute (#191)

[jakejackson1]

Closes #191

What

OtlTags::language() falls back from the language subtag to the language and region. It joined them with nothing in between, so it looked up zhhk, but the only region keys in Ucdn::$ot_languages are written with a hyphen (zh-cn, zh-hk, zh-mo, zh-sg, zh-tw, at src/Ucdn.php:778-782). No other key is four or more letters long, so the branch could never match. Every Chinese region laid text out with the script's DFLT entry.

-		} elseif ($lang != '' && $country != '' && isset(Ucdn::$ot_languages[$lang . $country])) {
-			// Never matches: the region keys are written 'zh-hk' (#191)
-			$langsys = Ucdn::$ot_languages[$lang . $country];
+		$region = $lang . '-' . $country;
+
 		if ($lang != '' && isset(Ucdn::$ot_languages[$lang])) {
 			$langsys = Ucdn::$ot_languages[$lang];
+		} elseif ($lang != '' && $country != '' && isset(Ucdn::$ot_languages[$region])) {
+			$langsys = Ucdn::$ot_languages[$region];

The fix belongs in the lookup key, not in Ucdn. The hyphenated keys are the table's own convention, generated like the rest of it.

How lang is split

language() splits the tag on - and lower-cases each part:

• the first subtag is the language
• the second is the region only if it is two letters long, so a four-letter script subtag like Hant is skipped
• a third subtag is always the region

So zh-HK, zh-hk and zh-Hant-HK all look up zh-hk. zh-Hant alone has no region and gets no language system. A language with its own key (e.g. sr) still wins over its region, as before.

Test font and shaping test

I checked every font in tests/data/ttf and packages/*/fonts for a language system starting ZH in GSUB or GPOS: none has one.

NotoSansTC-RegionLangSys-Synthetic.ttf makes the fix visible. It's Noto Sans TC 2.004 (OFL 1.1), subset in fontTools 4.59.2 to space, 骨, 一, 二 and 三. GPOS, BASE, STAT and the vertical metrics are dropped, and name IDs 1, 4 and 6 are renamed. It also gets a GDEF marking the four Han glyphs as bases, because mPDF refuses OpenType layout to a font without GDEF. Its GSUB is replaced by one script, hani, with no default features and one locl lookup per language system:

language system	骨 U+9AA8 is drawn as
ZHH	三 U+4E09
ZHS	一 U+4E00
ZHT	二 U+4E8C

ChineseRegionLangSysTest writes <p lang="…">骨</p> and records the glyph drawn through TextRecordingMpdf. It runs zh-HK, zh-TW and zh-CN, one per language system. I measured every row below the same way:

lang	before	after	hb-shape --language 14.3.1
zh-HK	骨	三 (ZHH)	ZHH ✓
zh-Hant-HK	骨	三 (ZHH)	ZHH ✓
zh-TW	骨	二 (ZHT)	ZHT ✓
zh-tw	骨	二 (ZHT)	ZHT ✓
zh-MO	骨	二 (ZHT)	ZHH (see below)
zh-CN	骨	一 (ZHS)	ZHS ✓
zh-SG	骨	一 (ZHS)	ZHS ✓

OtlTagsTest has a case for every row of the issue's table, plus zh-Hant-HK, lower case, and a region the table has no entry for (zh-US → DFLT).

Where HarfBuzz goes further: #201

HarfBuzz's hb_ot_tags_from_complex_language() (src/hb-ot-tag-table.hh) also maps the script subtag, and it checks the script before the region:

    if (lang_matches (&lang_str[1], limit, "zh-hans", 7))
      tags[0] = HB_TAG('Z','H','S',' ');  /* Chinese, Simplified */
    ...
    if (lang_matches (&lang_str[1], limit, "zh-hant", 7))
      tags[0] = HB_TAG('Z','H','T',' ');  /* Chinese, Traditional */

It also maps bare zh → ZHS, and Macao → ZHTM then ZHH. The issue asks for the region mapping, and mPDF has no equivalent for the rest, so I opened #201 instead of widening this PR. It covers zh, zh-Hant, zh-Hans, a script that disagrees with its region (zh-Hans-HK, zh-Hant-CN), and Macao. The zh-MO row above follows the issue and Ucdn's own 'zh-mo' => 'ZHT ' entry. OtlTagsTest::testChineseWithoutARegionHasNoLanguageSystem pins zh, zh-Hant and zh-Hans as they stand, with a reference to #201.

Fixtures

Regenerated from cold. I deleted tests/Mpdf/tmp/mpdf, tmp/mpdf and tmp/ttfontdata, then ran composer fontcache:update all, otldump:update all, shaping:update all and subset:update all. git status showed only the new font's five files. No existing fixture moved, which is expected because none of the golden masters sets a language. The new font's otldump lists hani with ZHH, ZHS and ZHT, each with locl. Its shaping master shapes with no language, so 骨 comes back unchanged.

Mutation check

With the key put back to $lang . $country and the caches cleared, 10 tests fail:

• all 3 data sets of ChineseRegionLangSysTest::testTheRegionSelectsTheLanguageSystem
• 7 of the 8 data sets of OtlTagsTest::testAChineseRegionSelectsItsLanguageSystem (all except zh-US)

Mirroring upstream

Upstream Otl::_getOTLLangTag() has the same key at src/Otl.php:6183 on development:

} elseif ($lang != '' && $country != '' && isset(Ucdn::$ot_languages[$lang . '' . $country])) {
	$langsys = Ucdn::$ot_languages[$lang . '' . $country];

The mirror changes $lang . '' . $country to $lang . '-' . $country in both places. Upstream splits the tag the same way (preg_split('/-/', …), region from a two-letter second subtag or any third subtag), so nothing else changes. The upstream PR also needs a CHANGELOG.md entry. The test font, ChineseRegionLangSysTest, and the language rows (as calls to _getOTLLangTag through reflection) can be carried over.

Base

Stacked on #200 (fix/192-indic-script-fallback) → #198 → #190 → #188 → #186 → gravitypdf. Before pushing I checked all five: all still open, none merged, so this targets fix/192-indic-script-fallback without a rebase.

/simplify

Applied:

• the region key is built once, into $region
• ChineseRegionLangSysTest now runs one region per language system (zh-HK, zh-TW, zh-CN), because OtlTagsTest already covers every row of the table without building an Mpdf for each

Skipped:

• dropping the $lang != '' guard on the region branch. It's redundant, but it was there before this change and it matches the branch above it.
• a shared drawn-codepoints helper on TextRecordingMpdf. That would mean editing about a dozen tests outside this diff.

Verification

• vendor/bin/phpunit: 2084 tests, 5955 assertions, 1 skipped (Fall back from a v2 Indic tag to its own original tag only (#192) #200 is 2065 / 5924 / 1).
• vendor/bin/phpunit --group=snapshot: OK, 79 tests, 110 assertions.
• composer cs: clean.
• phpstan: 33 errors, the same as the base, none in touched files.

🤖 Generated with Claude Code

[jakejackson1]

Rebased: #190 was squash-merged into gravitypdf (f9b3c0e), so this stack still carried its pre-squash commits. This branch is rebased onto its rebased parent with --onto; no conflicts and the base is unchanged. Suite, snapshots and cs pass on the new head, with no fixture movement.

[jakejackson1]

This PR merged into a base branch that had already been squash-merged, so its change never reached gravitypdf. It's carried to gravitypdf by #218.