Repository navigation
fix(updater): keep identifier leading zeros, keep published funds when a bulk feed fails - #87
Merged
Merged
Conversation
…n a bulk feed fails SPDR-2: Ticker, Identifier and SEDOL columns are no longer number-normalized, so CUSIP 037833100 and A-share ticker 002594 keep their leading zeros SPDR-3: a failed or empty product-data workbook or dividend-distribution feed keeps every published fund as published and reports it failed; only new funds are built
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
sheetToTablerannormalizeNumberTexton every cell, so text identifiers lost their leading zeros. The SSGA workbooks store Ticker, Identifier and SEDOL as text cells (t="s"), weights and amounts as numbers. Published damage today: SPY 40 CUSIPs (AAPL37833100instead of037833100, AMZN, BRK.B), GXC 295 A-share tickers (2594instead of002594,333instead of000333), SPAB 1 SEDOL; an E-shaped CUSIP like123456E10would have turned into a number. Now the Ticker, Identifier, SEDOL, CUSIP and ISIN columns keep the published text; every other column is normalized as beforeterValueand null index distributions. Now a failed or empty bulk feed is a failed required source: every fund that has ameta.jsonkeeps its published state and is reportedfailed(reason=product-data workbook failed, kept as published), only new funds are built. When every selected fund is published, the run ends red ("all N selected funds failed") with the index unchangedPublished values that change
TICKERS=SPY,GXC,SPAB REQUEST_SLEEP=0(exit 0, 3 updated): holdings diffs changed only identifier lines (SPY 40, GXC 295, SPAB 1, 0 other lines). The data was discarded (git checkout -- api/) because the same run also refreshed the day's NAV and history; the scheduled run corrects all fundsChecks
bun test: 41 pass, 0 fail, exit 0; the same run with the config controls exported also exits 0origin/main:parsingidentifier columns (002594,037833100,0263494,123456E10kept, Shares Held still expanded) andpipeline 3bulk feed (product-data and dividend-distribution each fail on a second run: SPY, XLK, GLD meta.json byte-identical, SPY index row unchanged, a new XLE fund is still built)bun build --target=bun scripts/update-data.ts --outfile=/dev/nullandsrc/main.tsx: ok.claude/tools/etfs/tc/check.sh: 48 lines before and after, no new errorscheck-readme.ts: ok;git diff --check: clean