Skip to content

fix(utils/format): keep integer digits past the 5 sig fig cap - #200

Open
pucedoteth wants to merge 1 commit into
nktkas:mainfrom
pucedoteth:fix/format-price-integer-exemption
Open

pucedoteth wants to merge 1 commit into
nktkas:mainfrom
pucedoteth:fix/format-price-integer-exemption

Conversation

@pucedoteth

Copy link
Copy Markdown

Problem

formatPrice truncates to MAX_DECIMALS - szDecimals places, then applies a 5 significant figure cap to anything still carrying a fraction. Once a price has more than 5 integer digits, that second step starts eating the integer digits:

formatPrice("123456.7", 0)    → "123450"    (want "123456")
formatPrice("1234567.8", 0)   → "1234500"   (want "1234567")
formatPrice("999999.9", 0)    → "999990"    (want "999999")
formatPrice("123456.789", 5)  → "123450"    (want "123456")
formatPrice("123456.7", 0, "spot") → "123450"

On a BTC-class price that is a $6.70 error on the first line and $67.80 on the second — a limit order resting materially away from where the caller asked.

It does not have to round that far. The docs are explicit, with this exact example:

Integer prices are always allowed, regardless of the number of significant figures. E.g. 123456 is a valid price even though 12345.6 is not.

The JSDoc on formatPrice already lists that rule, and the suite already asserts formatPrice("1234567", 0) === "1234567" — a 7 digit integer passing straight through. The exemption was only ever honoured when the value happened to already be whole after the decimal truncation; it was never used to keep precision when there was a fraction to drop.

Fix

Compute both valid forms — 5 significant figures, and the whole number — and keep whichever stayed closer to the input. Both are truncations toward zero, so that is whichever sits further from zero.

Prices with 5 or fewer integer digits are untouched, because 5 significant figures is at least as fine as a whole number there. formatPrice("12345.6", 0) is still "12345", formatPrice("1234.56", 0) still "1234.5".

Verification

deno task check green (fmt, lint, check --doc, jsdoc sync, export sync). deno test -A tests/utils/ tests/signing/: 114 steps, 0 failed. The format suite goes 45 → 47 steps, all passing.

Red/green — reverting only src/utils/_format.ts and keeping the tests fails the new step, while the companion step ("keeps sig figs when they are finer than a whole number", which pins behaviour that must not change) passes either way.

I also swept the property directly rather than trusting the examples. Over 2,168 combinations — perp and spot, szDecimals 0–8, nine mantissas across exponents 1e-8 to 1e8 — checking each output against an independent implementation of the documented rule:

outputs violating the rule 0
outputs exceeding the requested price 0
strictly more precise than before 98
less precise than before 0

So the change is strictly better where it differs and never invalid.

Note I have one earlier PR still open here (#195); this is unrelated and touches a different file.

🤖 Generated with Claude Code

formatPrice truncated to `MAX_DECIMALS - szDecimals` places and then, for
anything left with a fraction, to 5 significant figures. Once a price has
more than 5 integer digits that second step eats the integer digits:

    formatPrice("123456.7", 0)   -> "123450"   (want "123456")
    formatPrice("1234567.8", 0)  -> "1234500"  (want "1234567")

The docs are explicit that it does not have to: "Integer prices are always
allowed, regardless of the number of significant figures. E.g. `123456` is
a valid price even though `12345.6` is not." The JSDoc already listed that
rule, and the suite already asserted `formatPrice("1234567", 0)` passes a
7 digit integer straight through - it just was never reached when there
was a fraction to drop first.

Compare both valid forms and keep the one that stayed closer. Both are
truncations toward zero, so that is whichever sits further from zero.
Prices with 5 or fewer integer digits are untouched: 5 significant figures
is at least as fine as a whole number there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant