Select HarfBuzz's language system for a variant, script, region or extended language subtag, and for Chinese other than zh (#206) - #221
Merged
Conversation
OtlTags::script() walked the v2-to-original table from the run's own entry to the end, so a v2 tag the font did not offer reached the original tag of every script listed after it: Bengali was laid out with a font's deva lookups and the original specification's reordering, but Devanagari never reached beng. HarfBuzz asks for dev3, dev2 and deva for Devanagari (hb_ot_all_tags_from_script) and then DFLT, dflt and latn (hb_ot_layout_table_select_script), never another script's tag. The walk is now one lookup of the run's own original tag. NotoSansBengali-DevaScript-Synthetic makes it visible: a subset of Noto Sans Bengali with its GSUB replaced by a DFLT and a deva script, each with a 'locl' lookup giving KA a different glyph. KA drew deva's KHA; it now draws DFLT's GA, as hb-shape does. One existing fixture moves. NotoSansGurmukhiUI-GPOS71-Subset offers only gur2 and guru, so the golden master's Devanagari and Bengali runs used to reach guru and be reordered to the original specification. They now find no script and are left unshaped, as a run is in any font with no entry for its script. The font has no glyph for either run. Test font: NotoSansBengali-DevaScript-Synthetic (Noto Sans Bengali 3.011, OFL 1.1), built in fontTools 4.59.2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… what they cover (#192) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
) OtlTags::language() fell back from the language subtag to the language and region run together, 'zhhk', but the only region keys in Ucdn::$ot_languages are written with a hyphen, 'zh-hk'. The branch could never match, so zh-HK, zh-TW, zh-MO, zh-CN and zh-SG all laid text out with the script's DFLT entry. It now joins them with the hyphen. No font in tests/data/ttf or packages/ has a Chinese language system. NotoSansTC-RegionLangSys-Synthetic is a subset of Noto Sans TC whose GSUB offers hani with a 'locl' lookup under each of ZHH, ZHS and ZHT, giving the same character a different glyph under each. For every row of the issue's table and zh-Hant-HK, mPDF now draws the glyph hb-shape --language draws. No existing fixture moves: the golden masters set no language. HarfBuzz also reads the script subtag, takes ZHS for bare zh, and ZHTM then ZHH for Macao. That is #201, and OtlTagsTest pins zh, zh-Hant and zh-Hans as they stand. Test font: NotoSansTC-RegionLangSys-Synthetic (Noto Sans TC 2.004, OFL 1.1), built in fontTools 4.59.2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… ZHTM then ZHH for Macao (#201) OtlTags::language() read the language subtag, then the language and region. Nothing mapped a script subtag and bare zh had no key, so zh, zh-Hans and zh-Hant laid text out with the script's DFLT entry, and a script that disagreed with its region took the region's tag: zh-Hans-HK got ZHH, zh-Hant-CN got ZHS. Macao got ZHT. Chinese now follows HarfBuzz 14.3.1's hb_ot_tags_from_complex_language(): Traditional in Hong Kong or Macao keeps the region's tag, then Hans is ZHS and Hant is ZHT whatever the region, then a hk, mo or tw subtag gives ZHH, ZHTM then ZHH, or ZHT, and anything else is ZHS. language() tries each candidate against the script in turn, so its return value and Otl.php are unchanged. zh on its own now selects ZHS where a font offers it. No font in tests/data/ttf or packages/ has a Chinese language system other than the synthetic test fonts, and no golden master or snapshot sets lang="zh". NotoSansTC-MacaoLangSys-Synthetic is #191's font with 四 under a fourth language system, ZHTM. For every case in ChineseLangSysTest mPDF draws the glyph hb-shape --language draws. Test font: NotoSansTC-MacaoLangSys-Synthetic (Noto Sans TC 2.004, OFL 1.1), built in fontTools 4.59.2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… one place (#201) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tended language subtag, and for Chinese other than zh (#206) OtlTags::language() looked a tag up by its first subtag, with Chinese the one exception. HarfBuzz 14.3.1 reads more of it, and mPDF now tries the candidates in HarfBuzz's order (hb_ot_tags_from_language() and hb_ot_tags_from_complex_language()): 1. -fonnapa, -polyton, -arevmda, -provenc, -fonipa, -geok, -syre, -syrj or -syrn after any language 2. a retired tag read whole: art-lojban, i-hak, i-lux, i-navajo, no-bok, no-nyn, zh-min, zh-min-nan 3. ga-Latg IRT, mnw-TH MONT, ro-MD MOL then ROM 4. Chinese scripts and regions, for every Chinese language 5. an extended language subtag, zh-yue ZHH 6. the language subtag, trying each of its tags in turn Only subtags before the first singleton count, as in HarfBuzz. Ucdn::$ot_languages gains zh and the nineteen other Chinese languages HarfBuzz maps (yue ZHH, lzh ZHT, the rest ZHS), gives ga IRI then IRT, and adds nv as NAV then ATH. Hans makes yue and lzh ZHS; any other script or region leaves them their own tag, where zh and its other members take zh's rules. The five zh- region keys are removed: nothing has read them since #201, and zh-mo's ZHT was wrong. An extended language Ucdn has no key for keeps the language subtag's tags, so ar-afb stays ARA. HarfBuzz's table gives most extended languages their macrolanguage's tag and the rest their ISO 639-3 code; syncing the table is left to its own issue. A font offering only IRT now lays out lang="ga" with it, one offering MOL lays out ro-MD with it rather than ROM (DejaVu Sans and Serif offer both under latn), and NotoSans-Regular's NAV, IPPH and APPH now serve nv, -fonipa and -fonnapa. No golden master or snapshot sets a language, and no fixture moves. Noto-LanguageTags-Synthetic merges subsets of Noto Sans TC, Georgian, Myanmar and Syriac with a 'locl' lookup under each of PGR, IRT, MOL, MONT, PRO, SYRE, IPPH, KGE, ATH, ZHS, ZHT and ZHH. For every row of the issue's table mPDF now draws the glyph hb-shape --language draws, where before it drew the unsubstituted one or ZHS's. Test font: Noto-LanguageTags-Synthetic (Noto Sans TC 2.004, Noto Sans Georgian 2.005, Noto Sans Myanmar 2.107, Noto Sans Syriac 3.000, all OFL 1.1), built in fontTools 4.59.2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e() whether a language is Chinese (#206) The singleton cut becomes one preg_replace, the language comes off the front of the subtags so one array serves every rule, chinese() owns the test for a Chinese tag, and the extended language is looked up once. An empty tag returns before any of it, as it did before #206. Every expectation still matches hb_ot_tags_from_script_and_language(), the 87,094-tag comparison with HarfBuzz's own table differs only where it did, and each mechanism's mutation fails the same tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #206.
This carries PR #213's change to
gravitypdf. #213 was merged into its stacked base branch after that branch had already been squash-merged intogravitypdf, so its commits never reachedgravitypdfand #206 stayed open. The commits are #213's own, replayed withgit rebase --ontoand patch-for-patch identical pergit range-diff. See #213 for the full description, measurements and review.This branch also contains the #192, #191 and #201 recovery PRs, and is merged after them.
🤖 Generated with Claude Code