Read a binary dictionary's XML with a reader built for it

[ww-mw]

Reading a compressed-binary .sldd was 94% XML parsing — 3396 ms of a 3804 ms ingest, 22 MB/s over the 74.7 MB that 2.75 MB of zip inflates to. fast-xml-parser is a general engine paying for namespaces, DOCTYPE, CDATA, stop nodes, path matching and entity-expansion limits on every one of 6.9 M elements.

A dictionary's XML has a closed element vocabulary — across all 32 zipped dictionaries available (469.4 MB) there are five element names and nothing else. DictionaryXmlFast reads that subset in one pass and returns null for anything else, which the seam hands to the old engine.

The design, in one line

The fallback is the specification, not a feature flag. This file does not have to be a correct XML parser; it has to be a correct reader of the subset it recognises and provably silent on everything else. Every lenient behaviour XmlReader's contract pins — an unclosed tag that keeps its attributes, a mismatched close tag that is accepted, a non-DataSource root — is inherited by falling back rather than reimplemented. A bail is never wrong, only slower.

The one failure that rule cannot absorb is an answer that is non-null and disagrees. Four of those were found late, by looking for exactly them, and are the subject of the second half of this description.

Numbers

Both arms measured in separate cold processes with the compiled seam patched between them, never inferred by subtracting a delta.

	before	after
whole-file ingest, 74.7 MB dictionary	3804 ms	1062 ms	3.58x
parse alone	3396 ms (22 MB/s)	589–638 ms (117–127 MB/s)	5.3–6.1x
peak RSS	1237 MB	889 MB	−28%

The parse figure is a range because the probe was run three times — before the numeric-reference branch, after it, and after the line-ending normalization added below — and the ~3–8% spread is this machine, not the code. The slowest of the three clears the gate by a wide margin.

Flat ~120 MB/s across a 1000x size range, and the 0.2 MB dictionary is faster too, so there is no size below which the old path wins. Retained heap is unchanged (153 vs 159 MB) and necessarily so — the output tree is identical by design, so the saving is in peak, not in what survives.

XML parsing is no longer what reading a dictionary costs: unzip and the data-model walk are now the majority of the time rather than the 6% they used to be.

Why you can believe it is identical

Five checks, each blind where the others see:

1. Corpus differential — all 32 zipped dictionaries, 469.4 MB, 18,505,864 keys compared positionally so key insertion order is inside the claim, Object.is on primitives so 7 vs '7' is a divergence. 32/32 identical, 0 declines.
2. 89 unit tests. Every accepted case asserts three things: the written-down literal, the live engine's agreement with that literal, and key insertion order. Every declined case asserts null and that the engine still answers what it answered before this reader existed — including the inputs where the engine throws, because a caller reading null as "empty document" would turn a raised error into silent data loss.
3. 24 differential tests — all 19 binary .sldd fixtures, plus 6,000 seeded generated documents per run, asserting fast(xml) === null || deepEqual(fast(xml), generic(xml)) with key order, and that the engine did not throw.
4. Mutation testing, 39 mutants: 34 behaviour-changing all caught, 3 provably equivalent, 2 controls that must survive. It found four real holes in the tests, all closed.
5. An adversarial pass whose only brief was "find input that is non-null and wrong" — 216 hand-written cases and 10,000 generated documents, classifying every answer as match / decline / diverge / engine-threw-but-fast-accepted. It found four families, below. It now reports 0 problems.

Coercion is not reimplemented — strnum does it with the same options fast-xml-parser passes, so 007, 0x1F, 1.0 and an integer too large for a double land exactly where they already landed.

Six new MATLAB-written fixtures, and what they caught

Covering what the corpus could not: subclasses of Simulink.Parameter and Simulink.Signal, a twenty-property custom MCOS class, a custom enumeration, and the type-defining MathWorks classes as MATLAB actually writes them in binary rather than as a hand-authored fixture claims it does. Each is a twin pair — the same values written both compressed-binary and JSON text — so a difference between the two readers cannot be confused with a difference between the two formats.

They earned themselves immediately. A census over 469.4 MB had concluded that no dictionary contains a numeric character reference. That was a true statement about 32 files which I had read as a fact about the format, and it was wrong: MATLAB writes 
 for a CR inside a char property, and not one of those files held a multi-line char value. The first generated fixture had one, the differential test went red, and numeric references are now read as the literal text the engine also leaves standing (it does not decode them either — numericAllowed follows htmlEntities, which is off).

The decline had been correct — the output was never wrong — but a decline costs the whole file, so one carriage return in 74.7 MB would have paid the full 3358 ms. That is the fallback design converting a correctness bug into a performance cliff, working as intended and still worth fixing.

Only a reference to a codepoint XML 1.0 prohibits still declines, because the engine deletes those from the value and that table belongs to the engine. MATLAB cannot write one: asked to store a character XML forbids, it re-spells the whole property as Format="decimal" rather than emitting a reference it is not allowed to emit.

The four divergences the first four checks could not see

Checks 1–4 all ask the same question in different clothes: does this agree with the engine on input I thought of? None asks is there input where it answers and is wrong? So that sentence was made a brief of its own, and there were four — every one a non-null answer that disagreed.

input	this reader said	the engine says
a literal \r anywhere	'a\rb'	'a\nb'
<P Value\f="007" Value="b"/>	two keys, one named @_Value\f	one key, @_Value: 'b'
<?xmlfoo?>	a ?xml declaration	a PI named ?xmlfoo
102 nested <P>	a tree	throws Maximum nested tags exceeded

The CR is the one that matters, and it is now normalized. XML 1.0 §2.11 requires a parser to present \r\n and a lone \r to the application as \n; fast-xml-parser does it in one replace before it parses anything. A reader that builds its values by slicing the input string skips that step with no branch anywhere to suspect — the absence of a copy looks exactly like the fast path working. Fixed with the engine's own expression behind an indexOf('\r') guard: 4–7 ms on 74.7 MB (0.7% of the parse), and the copy itself never happens on real input, because all 32 corpus dictionaries contain no raw CR — MATLAB writes 
 precisely because this normalization exists.

The other three decline, and they sharpen the rule the file rests on. Knowing less than the engine is safe only while it makes this reader stop; it is a bug when it makes the reader continue on a narrower reading. isSpace is XML's four whitespace characters, which is the right definition for a reader, while the engine splits names on JavaScript's \s — so an attribute name carrying \f, \v, NBSP, a Unicode space or a BOM read as one attribute where the engine makes two and then folds a repeat last-wins, differing even in key count. <?xml is a name and not a prefix. And past maxNestedTags (100, which XmlReader does not override) the engine raises instead of answering, so a reader that answers there turns a loud failure into quiet data; the boundary was measured rather than copied — 101 opens parse, the 102nd throws, and a self-closing tag is not counted at all because the check sits in the opening-tag branch.

Why the earlier checks were blind is the part worth keeping: the corpus is MATLAB's own output and cannot contain a raw CR; the fuzz alphabet had no \r and no alien whitespace because I wrote it (the second time the alphabet was the limit); the unit tests pin what I believed, and I did not know XML normalized line endings; and mutation testing cannot find missing code — three of these four were absences, and "the normalization step does not exist" is not a mutation of anything.

All four are pinned against regression: 13 new unit cases, carriage returns and alien whitespace and a repeated attribute name and the <?xml-prefixed PIs added to the fuzz alphabet, and nine mutants that each revert one of these fixes, all nine caught, plus a tenth that correctly survives because removing the indexOf guard changes cost and not answers.

One known decline left alone: a BOM-prefixed part declines, which is correct but pays for the whole file. Unlike 
, the evidence says MATLAB does not write one — all 32 corpus dictionaries take the fast path.

Verification

npm run verify green: typecheck, build, smoke, 5,112 tests passed / 0 failed (was 4,999), check:pack, check:leak, check:browser. The corpus differential, the fixture differential, the mutation run and the adversarial probes were all re-run after the fixes above, not carried over from before them.

Not in this change

• readModelXml / readProjectXml are untouched — a .slx part's vocabulary is not closed and its documents are far smaller, so neither half of the argument applies there.
• Parse's share of open-to-paint in the extension. Everything measured here is this package in a bare node process; the claim is bounded to ingest deliberately.
• Format="decimal" is emitted faithfully by both readers and decoded by neither — a pre-existing gap in BinarySlddParser, found by these fixtures, left alone as out of scope.