fix(data): repair main CI after W36 publish (stale derived artifacts + backfill count) - #736
Merged
Merged
Conversation
…W36 publish Regenerate content/data pages, repository_summary.json, static/data/repositories.json, and star-velocity-explorer.json from the checked-in raw crawl artifacts using their owning scripts (generate_data_pages.py, generate_repository_summary.py, export_trend_explorer_data.py) so committed derived output matches source data after the W36 publish sync landed on main. The weekly topic backfill idempotency test hard-coded an expected count of 15 files (W21-W35). The W36 publish added a 16th weekly file, so the count is now derived from len(EXPECTED_TOPICS) instead of a literal, and W36's expected topics were added to that fixture map, verified against the deterministic backfill_weekly_topics() output for the current taxonomy registry. No prompt-injection sanitization logic was changed; sanitize_repo_content's canonical INJECTION_PHRASES warnings for legitimate MCP repo text mentioning "system prompt" are expected logging output, not failures. Fixes CI on main after commit 7b2fde7 (sync: publish data -> main #734). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
🔎 Site preview readyA rendered preview of this PR was built with Hugo extended (drafts + future content included). You can find it by opening workflow run → Artifacts → Download it, then serve and open it locally: gh run download 33629636687 --repo jmservera/SquadScope --name site-preview --dir site-preview && cd site-preview
python3 -m http.server 8080 # then open http://localhost:8080/(Links are root-relative, so open it via a local web server rather than Reviewer checklist
|
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The changes consistently update the W36-derived artifacts and adjust the affected test fixture in a self-contained way aligned with the stated CI-repair scope.
Pull request overview
This PR repairs main CI after the 2026-W36 publish sync by regenerating derived artifacts/pages that depend on the newly-added weekly data, and by updating the weekly-topic backfill test fixture to include W36 while removing a stale hard-coded weekly-file count.
Changes:
- Updated
tests/test_weekly_topic_backfill.pyto includeW36.mdinEXPECTED_TOPICSand to assert the backfill count vialen(EXPECTED_TOPICS)rather than a literal. - Regenerated data pages under
content/data/to reflect 2026-W36 (“as of” week/date, source ranges, and refreshed rankings). - Regenerated derived JSON exports used by the site/tools (repository summary + explorer datasets) to include the W36 raw artifact.
File summaries
| File | Description |
|---|---|
tests/test_weekly_topic_backfill.py |
Adds W36 expected topics and makes the backfill count assertion derive from the fixture map to avoid staleness. |
content/data/_index.md |
Bumps the data-pages index metadata to 2026-W36. |
content/data/top-ai-repositories-this-month/index.md |
Regenerated monthly top-AI ranking page for 2026-W36 (metadata + updated ranking rows). |
content/data/most-starred-mcp-projects/index.md |
Regenerated MCP ranking page for 2026-W36 (metadata + expanded/updated ranking rows). |
content/data/fastest-growing-ai-repositories-this-year/index.md |
Regenerated fastest-growing ranking page for 2026-W36 (metadata + updated ranking rows). |
data/observatory/repository_summary.json |
Regenerated repository summary artifact to include data/raw/2026-W36.json in provenance and records. |
static/data/repositories.json |
Regenerated repository explorer dataset (as-of week bumped to 2026-W36). |
static/tools/star-velocity-explorer.json |
Regenerated star-velocity explorer dataset to include W36 coverage. |
Review details
- Files reviewed: 4/8 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The changes are consistent with the stated CI-repair scope (update the backfill expectations for W36 and refresh derived data pages) and do not introduce new logic risk.
Review details
- Files reviewed: 4/8 changed files
- Comments generated: 0 new
- Review effort level: Lite
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Repairs pre-existing CI failures on
mainthat are independent of any other in-flight PR.mainat7b2fde7("sync: publish data → main (#734)") merged new crawl/analysis source data (including a new2026-W36weekly artifact) without regenerating the derived pages/exports that read it, and left a hard-coded weekly-file count stale.Root cause
content/data/*pages,data/observatory/repository_summary.json,static/data/repositories.json, andstatic/tools/star-velocity-explorer.jsonare generated from checked-in raw artifacts by owning scripts, but were not regenerated after the W36 publish sync landed.tests/test_weekly_topic_backfill.py::test_backfill_is_idempotent_and_assigns_expected_topicshard-coded an expected count of15weekly files (W21–W35). The W36 publish added a 16th weekly file (content/weekly/2026/W36.md), so the real count is now 16.These are one atomic CI-recovery concern: the backfill test's per-file assertions iterate
EXPECTED_TOPICS, so the count and the topic map for the new week have to land together for the test to pass at all.Fix
scripts/generate_data_pages.pyscripts/generate_repository_summary.py --from-crawlscripts/export_trend_explorer_data.pytests/test_weekly_topic_backfill.py:"W36.md"toEXPECTED_TOPICSwith the topics already committed incontent/weekly/2026/W36.md(AI Coding Agents,MCP Ecosystem,Developer Tools,Local First), verified by directly invokingbackfill_weekly_topics()against the current taxonomy registry — the output is deterministic and matches the existing frontmatter for every week, W21–W36.15tolen(EXPECTED_TOPICS)so this test derives its expected count from the fixture map instead of a literal that will go stale again on the next weekly publish.No prompt-injection sanitization behavior changed.
scripts/sanitize_repo_content.py's canonicalINJECTION_PHRASEScheck still logs warnings for ranking rows containing "system prompt" (legitimate MCP repo text) — that's expected logging output surfaced via--checksubprocess stderr, not a functional failure, and was not weakened.Validation
pytest tests/test_generate_data_pages.py::test_data_pages_are_regenerated_from_artifacts tests/test_repository_explorer.py::test_repository_summary_is_current_and_schema_valid tests/test_trend_explorer_tool.py::test_trend_explorer_export_is_current tests/test_weekly_topic_backfill.py→ 7 passed.pytest tests/→ 1693 passed, 1 skipped, 1 failed (test_public_json_schema_contracts.py::test_cost_summary_rejects_non_rfc3339_generated_at). Confirmed viagit stashthat this single failure is pre-existing onmainat7b2fde7before this change (ajsonschema/RFC3339 format-checker environment issue), unrelated to this repair's scope, and left untouched per the task's "don't fix pre-existing unrelated issues" rule.ruff check/ruff format --checkon the changed test file: clean.Scope note
This is a data/derived-artifact repair only — no product/source code, template, or documentation changes are mixed in, per repo generated-data hygiene rules.