fix: stop KaTeX swallowing currency in assistant messages - #1135
Open
philmerrell wants to merge 2 commits into
Open
philmerrell wants to merge 2 commits into
philmerrell wants to merge 2 commits into
Conversation
A message reading "Q1 ($100K), Q2 ($115K), and the total ($215K)" rendered
as "Q1 (100K),Q2(115K), total ($215K)" -- the sentence, not just the numbers.
streaming-text set ngx-markdown's `katex` attribute with no `katexOptions`,
so its DEFAULT_KATEX_OPTIONS applied. Those add `{ left: '$', right: '$' }`,
which KaTeX upstream deliberately leaves commented out in auto-render.js:
// LaTeX uses $...$, but it ruins the display of normal `$` in text
`renderMathInElement` pairs delimiters positionally inside a text node, so
any two amounts on one line become a formula. Bind explicit delimiters that
drop it (katex-delimiters.ts).
Removing it alone would have left NO working inline math, because `\(...\)`
never worked here either: `(` and `[` are CommonMark-escapable, so marked
strips the backslash before KaTeX ever runs --
marked.parse('Let \\(x^2\\) hold.') -> '<p>Let (x^2) hold.</p>'
and a system-prompt line asking the model for `\(...\)` does not reliably
override `$...$` (measured: Haiku 4.5 wrote `$e^{i\pi} + 1 = 0$` on the turn
the new instruction was live). So katex-math-markdown.ts adds marked inline
tokenizers that claim these spans before the escape rule can:
- `\(...\)` / `\[...\]` re-emitted verbatim so they reach the DOM at all
- `$...$` rewritten to `\(...\)`, but only where it cannot be
currency -- Pandoc's rule plus a digit check, so
"from $1M to $2M, i.e. $r = 2$" resolves exactly
right. Bare `$` stays out of KATEX_DELIMITERS, which
is what keeps KaTeX from re-pairing dollars in the
DOM and undoing that decision.
Two further pre-existing breakages, found while hunting edge cases: `$$`
survived markdown but its CONTENTS did not.
- `$$a*b*c$$` -> `$$a<em>b</em>c$$`. Emphasis deletes the asterisks and splits
the text node in three; splitAtDelimiters works within one text node, so the
`$$` stop pairing and nothing renders at all.
- `$$\begin{pmatrix} a \\ b \end{pmatrix}$$` -> `\\` collapses to `\`, breaking
every matrix and every multi-row alignment. Same for a bare `\begin{...}`.
Both fixed by giving `$$...$$` and `\begin{env}...\end{env}` their own
tokenizers, so marked's inline rules never touch the body.
Note that escaping is NOT an alternative: a `$` is decoded to a literal
`$` when marked's output is assigned to innerHTML, which happens before KaTeX
walks the DOM. The entity form breaks identically -- verified against the
app's own KaTeX build.
Tests: streaming-text.katex.spec.ts renders end-to-end through the real
pipeline (marked -> Angular sanitizer -> KaTeX), with a guard test asserting
the globals are loaded so a "no math rendered" expectation cannot pass for the
wrong reason. angular.json's `scripts` are emitted but not evaluated in the
test build, hence the explicit imports. SPA suite: 3280 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RESPONSE GUIDELINES told the model to write non-math uses of `$` as `$`. That rule cost tokens on every turn and did nothing: marked passes the entity through to innerHTML, the browser decodes it to a literal `$` in the text node, and KaTeX walks the DOM after that -- so the entity form produced byte-identical breakage. Verified against the app's own KaTeX build. What it did accomplish was leaking the 9-character string "$100K" into generated .pptx/.xlsx cells (fixed in an earlier commit by scoping the rule to chat markdown, which left the useless rule itself in place). The real fix is in the SPA, where `$...$` now resolves by context rather than by positional pairing. So the guidance simply states what renders: `$...$` or `\(...\)` inline, `$$...$$` or `\[...\]` for display, currency as a plain `$`. Deliberately permissive rather than prescriptive -- steering the model off `$...$` was measured and does not hold, so the renderer handles it instead. Net effect on the cacheable prefix: four lines shorter, and it re-writes once per session on rollout. Tests: TestKatexGuidance pins the entity out of the prompt. Backend suite: 8633 passed, 3 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Reported on a real conversation. The assistant wrote:
and the SPA rendered:
The prose between the amounts disappeared, not just the numbers.
Cause
streaming-text.component.tsenabled ngx-markdown'skatexplugin with nokatexOptions, so itsDEFAULT_KATEX_OPTIONSapplied. Those add{ left: '$', right: '$' }— which KaTeX upstream deliberately leavescommented out in
auto-render.js, with the reason attached:renderMathInElementpairs delimiters positionally within a text node, soany two currency amounts on one line become one formula.
Why the obvious fixes don't work
Three things looked like answers and are not. Each is documented in the code
so nobody re-derives them:
$does nothing. The system prompt already told the model to escape$as an HTML entity. marked passes the entity intoinnerHTML, the browserdecodes it to a literal
$in the text node, and KaTeX walks the DOM afterthat — the entity form breaks identically. Verified against the app's own
KaTeX build. It only ever succeeded at leaking
$100Kinto generated.pptx/.xlsx cells (#1126's follow-on). Rule deleted.
\(…\)was never an available fallback.(and[areCommonMark-escapable, so marked strips the backslash before KaTeX runs:
So
$…$and$$…$$were the only working delimiters. Dropping$…$alonewould have left no inline math at all.
Telling the model to write
\(…\)doesn't hold. Measured, with the newprompt confirmed live: Haiku 4.5 wrote
$e^{i\pi} + 1 = 0$anyway.$…$istoo dominant in training data for one prompt line to override.
The fix
shared/utils/katex-math-markdown.ts— marked inline tokenizers registered atbootstrap, which claim math spans before marked's escape and emphasis rules
reach them:
\(…\)/\[…\]$…$\(…\), only where it cannot be currency$$…$$\begin{env}…\end{env}shared/utils/katex-delimiters.tsbinds explicit delimiters with bare$removed. That is what makes the guarded rule safe: KaTeX never re-pairs
dollars in the DOM, so it cannot undo the decision made with full context.
The currency test is Pandoc's plus a digit check — opening
$not followed bywhitespace or a digit, closing
$not preceded by whitespace nor followed bya digit, no line break.
Revenue rose from $1M to $2M, i.e. $r = 2$.resolvesexactly right: both amounts skipped, only
$r = 2$becomes math.Two more breakages found while hunting edge cases
Both pre-existing, both inside
$$…$$— the delimiter survived markdown, itscontents did not:
$$a*b*c$$$$a<em>b</em>c$$→ renders nothing$$\begin{pmatrix} a \\ b \end{pmatrix}$$\\→\→ broken matrixThe emphasis case is the subtle one:
<em>splits the text node in three, andsplitAtDelimitersworks within a single text node, so the$$stop pairingentirely. The
\\case broke every matrix and every multi-row alignment.Verification
Reproduced and re-verified in the running app against dev data — the reported
conversation renders correctly, and a fresh turn produced inline math, display
math, a 2×2 matrix and two currency amounts all correct in one response
(2 inline + 2 display KaTeX nodes, 0 errors, 0 leftover literal
$…$).streaming-text.katex.spec.tsrenders end-to-end through the real pipeline(marked → Angular sanitizer → KaTeX) rather than a mock. It includes a guard
test asserting the KaTeX globals are loaded, because angular.json's
scriptsare emitted but not evaluated in the test build — without it every "no math
rendered" expectation would pass for the wrong reason.
tsc -p tsconfig.app.json --noEmitcleanPrompt-cache note
DEFAULT_SYSTEM_PROMPTchanges, so the cacheable prefix re-writes once persession on rollout. It is four lines shorter afterwards.
🤖 Generated with Claude Code