Choose an Indic run's shaper on the script chosen for it, as HarfBuzz does (#199) - #222
Merged
Merged
Conversation
… does (#199) Otl::applyOTL() sent a run whose script has a shaper of its own to that shaper under any GSUB script, DFLT and latn included, and skipped a run the font offered no script for entirely. HarfBuzz decides the other way round in hb_ot_shaper_categorize(): under DFLT, or latn picked as the last fallback, the default shaper lays the run out with the font's features alone, and with no script chosen the script's shaper still runs. shaperForScriptTag() now makes that choice once the tags are known, per shaper as HarfBuzz 14.3.1 has it: - Indic, Sinhala and the South East Asian scripts (Indic and USE in HarfBuzz), and N'Ko and Mandaic: the default shaper under DFLT or latn - Myanmar: also under mymr, which replaces the mym2-only check - Syriac: the default shaper under DFLT only - Arabic, Khmer, Thai and Lao: their own shaper whatever was chosen A run with no script reaches its shaper as long as the document opened the script to OpenType layout (useOTL 0x80, which covers every script with a shaper). It is shaped with no lookups: loadGsubData() reads no GSUB table for it, which the font need not have, and the GSUB rule walks read the font's features for a script and language through gsubFeatures(), which has none to give. Measured with Noto Sans Bengali subsets whose GSUB has one 'locl' lookup, KA + I and KA + VIRAMA + SSA + I: DFLT and deva 09BF 0997 / 0997 09CD 09BF 09B7 -> 0997 09BF / 0997 09CD 09B7 09BF guru only 0995 09BF / 0995 09CD 09B7 09BF -> 09BF 0995 / 0995 09CD 09BF 09B7 which is what hb-shape draws. 288 lines of the shaping golden master move over 31 fonts, all of them now what hb-shape draws on the same font and text, 18 of them before as well: those 18 are an Arabic run gaining a kashida point on its tatweel in fonts with no Arabic script, and draw the same. The rest are the Devanagari, Bengali and Gurmukhi runs, reordered in the 8 fonts with no script for them and no longer in the fonts laying them out under DFLT or latn, and the Khmer run reordered in the 8 fonts with no Khmer script. The South East Asian shaper is now reached only under a Tai Tham tag, and Lanna Alif offers latn, so SeaTest and the Tai Tham snapshot, which pin a cluster reaching its reorderer through a 'ccmp' ligature, move to NotoSansTaiTham-LanaScript-Synthetic. The snapshot's broken cluster loses the dotted circle Lanna Alif drew; the new font has none. Test fonts: NotoSansBengali-GuruScript-Synthetic (Noto Sans Bengali 3.011, OFL 1.1) and NotoSansTaiTham-LanaScript-Synthetic (Noto Sans Tai Tham 2.002, OFL 1.1, script retagged lana, its Sakot ligatures moved from liga to ccmp), built in fontTools 4.59.2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… GSUB and GPOS features one way (#199) loadGsubData() took a flag only to return between its two halves: the derived tables every shaper reads, and the table, coverage and lookup list only a run with features needs. They are now two methods, and applyGSUB() calls the second where there are features. The GPOS feature read beside the GSUB one goes through the same accessor, and shaperForScriptTag() takes the shaper it narrows as an argument rather than reading it back off the object. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… DFLT script (#199) #175's synthetic font reached gravitypdf after this branch was written. Its GSUB offers DFLT and no Bengali or Gurmukhi script, so those two runs now take the default shaper and are left in logical order. hb-shape 14.3.1 does the same. Clusters cannot show it, since this font has no glyphs for these scripts and HarfBuzz keeps a reordered syllable's clusters apart, so it was checked on a font that has them: NotoSansBengali-GuruScript-Synthetic, retagged to this font's scripts, draws KA VIRAMA SSA I in logical order under DFLT and reordered where the font offers no script at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… DFLT script (#199) #170's synthetic font reached gravitypdf after this branch was written. Its GSUB offers DFLT and arab and no Indic script, so the Devanagari, Bengali and Gurmukhi runs take the default shaper and stay in logical order. hb-shape 14.3.1 does the same. Clusters cannot show it, since this font has no glyphs for these scripts and HarfBuzz keeps a reordered syllable's clusters apart, so it was checked on a font that has them: NotoSansBengali-GuruScript-Synthetic, retagged to this font's scripts, draws KA VIRAMA SSA I in logical order under DFLT and reordered where the font offers no script at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… moves their fonts' fixtures
…s add under #199's shaper choice (#199) Five synthetic fonts reached this branch from #189, #193, #201, #204 and #206 after its fixtures were recorded. Where a font's GSUB offers DFLT or latn and no script for the run, the run takes the default shaper and stays in logical order; where it offers no script that applies, as the hani-only NotoSansTC-MacaoLangSys-Synthetic does for Devanagari, Bengali and Gurmukhi, the Indic shaper still reorders it. Khmer keeps its own shaper whatever was chosen. hb-shape 14.3.1 agrees on every moved run. These fonts have no glyphs for the scripts, so it was checked on NotoSansBengali-GuruScript-Synthetic retagged to each font's scripts: logical order under DFLT and under DFLT with latn, reordered under hani alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ge base to compare against
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #199.
This carries PR #214's change to
gravitypdf. #214 was merged into its stacked base branch after that branch had already been squash-merged intogravitypdf, so its commits never reachedgravitypdfand #199 stayed open. The commits are #214's own, replayed withgit rebase --ontoand patch-for-patch identical pergit range-diff. See #214 for the full description, measurements and review.This branch also contains all seven other recovery PRs, plus a commit updating five fonts' shaping fixtures for this PR's shaper choice, and is merged after them.
New since #214:
4c394d4and3e1b81aupdate shaping fixtures for synthetic fonts that reached this branch from #170, #189, #193, #201, #204 and #206 after #214 was written.3e1b81acovers five fonts. Each moved run was checked against hb-shape 14.3.1. Those fonts have no glyphs for the scripts, so the check retaggedNotoSansBengali-GuruScript-Syntheticto each font's scripts. HarfBuzz draws KA VIRAMA SSA I in logical order underDFLTand underDFLTwithlatn, and reorders it underhanialone. mPDF agrees on every moved run. Two commit messages that had cited hb-shape's clusters as evidence were corrected, because HarfBuzz doesn't merge a reordered syllable's clusters. On the combined tree: 2301 tests, snapshots 79/109, cs clean, and a cold regeneration of all four golden masters leavestests/dataunchanged.🤖 Generated with Claude Code