Skip to content

fix(updater): keep identifier leading zeros, keep published funds when a bulk feed fails - #87

Merged
daggerok merged 1 commit into
mainfrom
fix/updater-review-highs
Oct 26, 2026
Merged

daggerok merged 1 commit into
mainfrom
fix/updater-review-highs

Conversation

@daggerok

Copy link
Copy Markdown
Owner

What

  • SPDR-2 (review HIGH): sheetToTable ran normalizeNumberText on every cell, so text identifiers lost their leading zeros. The SSGA workbooks store Ticker, Identifier and SEDOL as text cells (t="s"), weights and amounts as numbers. Published damage today: SPY 40 CUSIPs (AAPL 37833100 instead of 037833100, AMZN, BRK.B), GXC 295 A-share tickers (2594 instead of 002594, 333 instead of 000333), SPAB 1 SEDOL; an E-shaped CUSIP like 123456E10 would have turned into a number. Now the Ticker, Identifier, SEDOL, CUSIP and ISIN columns keep the published text; every other column is normalized as before
  • SPDR-3 (review HIGH): a failed product-data workbook or dividend-distribution bulk feed was swallowed ("continuing without it") and every refreshed fund was rewritten with null ISIN, CUSIP and SEC yield, an indicated instead of official dividend yield (basis code changed), the gross TER as terValue and null index distributions. Now a failed or empty bulk feed is a failed required source: every fund that has a meta.json keeps its published state and is reported failed (reason=product-data workbook failed, kept as published), only new funds are built. When every selected fund is published, the run ends red ("all N selected funds failed") with the index unchanged
  • README: one sentence for each rule under Metrics and caveats

Published values that change

  • On the next refresh of each fund: identifier cells regain their leading zeros, nothing else in the holdings moves. Scoped real run TICKERS=SPY,GXC,SPAB REQUEST_SLEEP=0 (exit 0, 3 updated): holdings diffs changed only identifier lines (SPY 40, GXC 295, SPAB 1, 0 other lines). The data was discarded (git checkout -- api/) because the same run also refreshed the day's NAV and history; the scheduled run corrects all funds
  • SPDR-3 changes values only on a run where a bulk feed fails

Checks

  • bun test: 41 pass, 0 fail, exit 0; the same run with the config controls exported also exits 0
  • Both new tests fail on origin/main: parsing identifier columns (002594, 037833100, 0263494, 123456E10 kept, Shares Held still expanded) and pipeline 3 bulk feed (product-data and dividend-distribution each fail on a second run: SPY, XLK, GLD meta.json byte-identical, SPY index row unchanged, a new XLE fund is still built)
  • bun build --target=bun scripts/update-data.ts --outfile=/dev/null and src/main.tsx: ok
  • .claude/tools/etfs/tc/check.sh: 48 lines before and after, no new errors
  • check-readme.ts: ok; git diff --check: clean

…n a bulk feed fails

SPDR-2: Ticker, Identifier and SEDOL columns are no longer number-normalized, so CUSIP 037833100 and A-share ticker 002594 keep their leading zeros
SPDR-3: a failed or empty product-data workbook or dividend-distribution feed keeps every published fund as published and reports it failed; only new funds are built
@daggerok
daggerok merged commit 766f78d into main Oct 26, 2026
1 check passed
@daggerok
daggerok deleted the fix/updater-review-highs branch October 26, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant