fix: resolve 2026-07-15 audit findings (retry-loop fail-closed, security, SEO, dark mode)#9642
Merged
Merged
Conversation
…ary list, gate agent triggers - impl-generate.yml: failure counter fails closed (API error = retry cap reached) and sums per-page counts — an unreadable count previously reported 0 prior failures and re-dispatched forever (issue #1010, ~1,200 runs in 38h) - spec-create.yml: replace quoted heredocs that shipped literal $SPEC_CONTENT into PR/issue bodies with unquoted heredocs (escaped backticks) + --body-file; list all 15 generate labels - impl-merge.yml: auto-close reads SUPPORTED_LIBRARIES from core/constants.py instead of a stale 11-library copy - ISSUE_TEMPLATEs: drop auto-applied trigger labels so a maintainer (triage rights) reviews issues before write-privileged Claude agents consume them - composite actions: SHA-pin setup-node/r-lib/julia refs; cover the action directories in dependabot's github-actions entry Audit 2026-07-15: Critical#1, High#3/#4/#7, Medium#13/#15. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
…upsert, bot SEO surface - core/database/connection.py: use the Cloud SQL connector's async path (create_async_engine + async_creator/asyncpg) instead of wrapping a sync pg8000 engine in async_sessionmaker; close via close_async() - api/mcp/server.py: guard NULL impl_tags in search_specs_by_tags - api/exceptions.py: DatabaseQueryError returns a generic client message; raw SQLAlchemy detail stays in server-side logs - api/cache.py: prune per-key locks when a factory raises; hold strong references to background refresh tasks (GC could jam refresh) - automation/scripts/sync_to_postgres.py: language/library seeds upsert (on_conflict_do_update) so registry updates reach the prod DB - alembic: add impl check constraints migration (quality_score range, review verdict) with pre-normalization and real downgrade - api/routers/seo.py: site-level JSON-LD + richer homepage for bots; /seo-proxy/plots and /seo-proxy/specs link every spec hub (crawlable internal-link graph); cached spec index with invalidation - api/main.py: CORS origins from settings.cors_origins (adds the promised www.anyplot.ai origin) - core/palette.py: first unit-test coverage (pool contract, semantic anchors, lazy cmaps, matplotlib-free import) Audit 2026-07-15: High#6/#10/#11, Medium#1/#2/#3/#4/#18/#22/#28/#40, Low. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
…ers, node 22 - shared overlayButtonSx/overlaySurfaceColor helpers in theme/tokens.ts; ImageCard/SpecOverview/ImagesGrid overlay controls follow dark mode (pattern extracted from SpecDetailView) - MuiMenu/MuiPopover paper wired to var(--bg-elevated)/var(--ink) with a dark-only hairline border - a11y: card actions reveal on :focus-within and stay visible on touch; focus-visible outlines on NavBar search/links and ThemeToggle; 44px tap targets on mobile - palette drift: Tailwind green and Python-blue replaced with imprint hues (StatsPage, SpecOverview) - Layout/useThemeMode context values memoized (stable identities) - useAnalytics.ts comments translated to English - nginx: Google-InspectionTool/GoogleOther route to bot pages; new security-headers.conf (CSP, nosniff, X-Frame-Options, Referrer-Policy, HSTS) included in both server blocks and add_header locations - Dockerfile: node:20-alpine (EOL 2026-04) -> node:22-alpine - robots.txt: document intended AI-crawler policy vs the Cloudflare managed block that currently contradicts it Audit 2026-07-15: High#1/#2, Medium#7/#10/#11/#12/#16/#21/#41, Low. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
… path and timeout message
- scripts/evaluate-plot.py: resolve plot-{light,dark}.png/html via
ANYPLOT_THEME (legacy plot.png fallback), run impls with the theme env
set, theme-correct blank check; static evaluator+rubric moved to a
system prompt with cache_control ephemeral (was re-billed per call)
- agentic/workflows/modules/agent.py: save_prompt/execute_template use
the module-level project root (artifacts landed in agentic/agentic/
runs/); timeout error derives its duration from the actual timeout
Audit 2026-07-15: High#5, Medium#25/#30, Low.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
…sport, repo health
- prompts (plot-generator, workflow-prompts README, impl-similarity,
impl-repair) and docs (/prime, copilot-instructions, project-guide)
reflect the 15-library / 4-language registry; repair prompt says
attempt {ATTEMPT}/4 matching impl-repair.yml; workflow inventory
matches .github/workflows/; plot.png code examples are theme-aware
- docs/reference/plausible.md: remove potd_dismiss + PlotOfTheDay
nav_click sources that no longer exist (31 events)
- README.md: Streamable HTTP is the only MCP transport (SSE endpoint
was removed)
- scrub leaked personal file:// paths from a palette doc
- add .editorconfig and PR template; remove stray _run_plotly.py
scratch file from plots/contour-3d
- CHANGELOG.md: audit-resolution entries under [Unreleased]
Audit 2026-07-15: Medium#14/#24/#26/#27/#29/#31/#32/#33, Low.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses the 2026-07-15 audit by hardening the impl-generation retry cap to fail-closed (stopping self-amplifying dispatch loops), tightening CI/security surfaces (template trigger gating, SHA pinning, nginx security headers), and bringing the product surface up to date across SEO bot pages and frontend dark-mode/a11y fixes—alongside new unit coverage for previously untested core behavior.
Changes:
- Fixes the impl-generate retry counter to fail closed and correctly count markers across paginated issue comments.
- Improves runtime correctness and security posture across backend (Cloud SQL async connector, exception sanitization, cache lifecycle) and frontend (dark-mode overlays, dropdown surfaces, focus/tap target a11y).
- Updates docs/prompts/workflows to reflect the 15-library / 4-language reality and adds missing tests + migrations for previously drifted constraints.
Reviewed changes
Copilot reviewed 58 out of 58 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/core/test_palette.py | Adds first unit tests for core/palette.py contracts and lazy cmap behavior. |
| tests/unit/core/database/test_connection.py | Updates tests to cover async Cloud SQL connector creation and async close. |
| tests/unit/automation/scripts/test_sync_to_postgres.py | Adds assertions for registry seed upsert semantics. |
| tests/unit/api/test_seo_helpers.py | Adds coverage for spec index generation and homepage JSON-LD/body copy. |
| tests/unit/api/test_routers.py | Extends SEO proxy router tests (JSON-LD, DB-backed spec counts, spec hub links). |
| tests/unit/api/test_exceptions.py | Verifies database error detail is not reflected in client messages but still logged. |
| tests/unit/api/test_cache.py | Adds tests for lock pruning on factory failure and background refresh task retention. |
| tests/unit/api/mcp/test_tools.py | Covers NULL impl_tags handling in MCP tag filtering. |
| scripts/evaluate-plot.py | Makes the local evaluator theme-aware and prompt-cache friendly; aligns with pipeline outputs. |
| README.md | Updates MCP transport documentation (Streamable HTTP only; SSE removed). |
| prompts/workflow-prompts/README.md | Updates workflow prompt docs for 15 libraries / derived language handling. |
| prompts/workflow-prompts/impl-similarity-claude.md | Updates similarity audit instructions for multi-language metadata layout. |
| prompts/workflow-prompts/impl-repair-claude.md | Updates repair attempt messaging to 4 attempts. |
| prompts/plot-generator.md | Updates generator prompt for 15 libraries, including JS harness constraints. |
| plots/contour-3d/implementations/python/_run_plotly.py | Removes the helper runner script (see PR comment for runtime impact). |
| docs/reference/plausible.md | Removes obsolete POTD-dismiss analytics events and updates totals. |
| docs/reference/palette-variants/v2/expert-reviews.md | Scrubs leaked local file:// references from documentation. |
| core/database/connection.py | Implements async Cloud SQL connector path (asyncpg + async_creator) and async close. |
| CHANGELOG.md | Documents audit fixes under [Unreleased] across Fixed/Security/Added/Changed. |
| automation/scripts/sync_to_postgres.py | Changes language/library seeding to upsert so registry edits propagate to existing rows. |
| app/src/theme/tokens.ts | Introduces reusable theme-aware overlay button styles. |
| app/src/theme/components.ts | Forces Menu/Popover surfaces to use theme CSS tokens in dark mode. |
| app/src/sections/spec-detail/SpecOverview.tsx | Improves dark-mode overlays and a11y (focus reveal, touch visibility); palette alignment. |
| app/src/sections/spec-detail/SpecDetailView.tsx | Deduplicates overlay button styling via shared helper. |
| app/src/sections/plots-gallery/ImagesGrid.tsx | Makes loading pill overlay theme-aware. |
| app/src/sections/plots-gallery/ImageCard.tsx | Makes copy overlay theme-aware + accessible for focus/touch users. |
| app/src/pages/StatsPage.tsx | Replaces off-palette green with imprint brand green RGBA. |
| app/src/layouts/NavBar.tsx | Improves tap targets and restores focus-visible outlines after all: unset. |
| app/src/layouts/Layout.tsx | Memoizes context provider values to avoid unnecessary rerenders. |
| app/src/hooks/useThemeMode.ts | Memoizes theme mode context value object for stable identity. |
| app/src/hooks/useAnalytics.ts | Translates inline comments to English; clarifies URL-building behavior. |
| app/src/components/ThemeToggle.tsx | Improves tap target sizing and focus-visible outline for accessibility. |
| app/security-headers.conf | Adds centralized nginx security header/CSP snippet for frontend. |
| app/public/robots.txt | Documents intended crawler policy and Cloudflare-managed override caveat. |
| app/nginx.conf | Adds bot UA mappings and includes security headers where add_header is used. |
| app/Dockerfile | Updates builder image to Node 22 and ships the security headers snippet into nginx image. |
| api/routers/seo.py | Adds site-level JSON-LD, richer bot homepage, and DB-backed spec hub link lists. |
| api/mcp/server.py | Hardens tag filtering against NULL impl_tags. |
| api/main.py | Restores CORS origin list to config-driven settings. |
| api/exceptions.py | Avoids reflecting raw DB error detail to clients while logging it server-side. |
| api/cache.py | Adds lock pruning on factory failure and strong refs for background refresh tasks. |
| alembic/versions/f4b8d2c6a9e1_add_impl_check_constraints.py | Adds missing DB check constraints with legacy-row normalization. |
| agentic/workflows/modules/agent.py | Fixes run artifact pathing and improves timeout error messaging. |
| agentic/docs/project-guide.md | Reconciles docs with 15-library pipeline and themed outputs. |
| agentic/commands/prime.md | Updates project overview to 15 libraries across 4 languages. |
| .github/workflows/spec-create.yml | Fixes heredoc expansion for bodies and updates generate label list. |
| .github/workflows/report-validate.yml | Documents manual label gate for report validation workflow. |
| .github/workflows/impl-merge.yml | Reads supported library list from core.constants.SUPPORTED_LIBRARIES to prevent drift. |
| .github/workflows/impl-generate.yml | Fixes retry counter pagination and fails closed on count API errors. |
| .github/pull_request_template.md | Adds a PR template with changelog/docs checklist. |
| .github/ISSUE_TEMPLATE/request-new-plot.yml | Removes auto-applied trigger labels; requires maintainer labeling. |
| .github/ISSUE_TEMPLATE/report-plot-issue.yml | Removes auto-applied trigger labels; requires maintainer labeling. |
| .github/dependabot.yml | Ensures Dependabot scans composite action directories for SHA pin updates. |
| .github/copilot-instructions.md | Updates Copilot repo instructions for 15-library / multi-language structure. |
| .github/actions/setup-r/action.yml | Pins setup-r action to a commit SHA. |
| .github/actions/setup-node/action.yml | Pins setup-node action to a commit SHA. |
| .github/actions/setup-julia/action.yml | Pins setup-julia and cache actions to commit SHAs. |
| .editorconfig | Adds repo-wide editor defaults aligned with ruff and frontend formatting. |
Comments suppressed due to low confidence (1)
plots/contour-3d/implementations/python/_run_plotly.py:1
- This file is being deleted, but
plots/contour-3d/implementations/python/plotly.pystill doesexec(Path(__file__).parent.joinpath("_run_plotly.py").read_text())(plotly.py:10). After this change, the Plotly implementation will raise FileNotFoundError at runtime. Either keep_run_plotly.pyin the repo, or (preferred) rewriteplotly.pyto be self-contained so this helper can be safely removed.
…ng helper The deleted _run_plotly.py was not a stray scratch file — plotly.py exec()'d it as a workaround for the plotly-package name collision. Inline the implementation with the canonical sys.path self-import guard used by the other 70+ plotly.py files (flagged by Copilot review on #9642). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
MarkusNeusinger
enabled auto-merge (squash)
July 16, 2026 13:02
Comment on lines
+528
to
+532
| body = f"<h1>plots</h1><p>{description}</p>{_spec_links_html(await _get_spec_index(db))}" | ||
| return HTMLResponse( | ||
| _render_bot_html( | ||
| title="plots | anyplot.ai", | ||
| description="Browse and filter visualization examples across 15 libraries in Python, R, Julia, and JavaScript: matplotlib, seaborn, plotly, bokeh, altair, plotnine, pygal, lets-plot, ggplot2, Makie.jl, Chart.js, D3.js, ECharts, Highcharts, MUI X Charts.", | ||
| description=description, |
Comment on lines
+549
to
+553
| body = f"<h1>specs</h1><p>{description}</p>{_spec_links_html(await _get_spec_index(db))}" | ||
| return HTMLResponse( | ||
| _render_bot_html( | ||
| title="specs | anyplot.ai", | ||
| description="Browse all Python plotting specifications alphabetically.", | ||
| description=description, |
Comment on lines
+84
to
+86
| title={ | ||
| "text": "contour-3d · plotly · anyplot.ai", | ||
| "font": {"size": 28, "color": INK}, |
7 tasks
MarkusNeusinger
pushed a commit
that referenced
this pull request
Jul 16, 2026
The #9644 entry insertion consumed the following block's "### Fixed" header, misfiling the #9642 fixes under "### Added". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kWZJgpARSK68RtfeHMEsK
MarkusNeusinger
added a commit
that referenced
this pull request
Jul 17, 2026
… code-blob over-fetch, CoC) (#9644) ## Summary - **Tag search finally uses the GIN index** — `SpecRepository.search_by_tags` cast the JSONB `tags` column to text and ran LIKE, which `ix_specs_tags` can never serve (seq scan on every MCP tag search) and which let `%`/`_` in tag values wildcard-match. On PostgreSQL it now emits per-category JSONB containment (`tags @> '{"<category>": ["<tag>"]}'` via `type_coerce`, keeping the left side a bare column so the index stays applicable); the SQLite test fallback escapes LIKE metacharacters via `autoescape` (audit 2026-07-15 Medium#17). - **Hot listing paths stop dragging the full code corpus through the DB** — MCP `list_specs` / `search_specs_by_tags` and `/libraries/{id}/images` all called `get_all_with_code()` (every code blob of ~3,240 impls per request). The MCP tools now resolve code *presence* with a lightweight id probe (`ImplRepository.get_ids_with_code`), the images endpoint fetches only its own library's code (`get_by_library_with_code`), and the now caller-less `get_all_with_code()` is removed (audit Medium#5, issue #7696). - **Feedback rate limiting no longer trusts a client-controlled header entry** — `/feedback` keyed its per-IP limit and duplicate suppression on the *first* `x-forwarded-for` entry, so a caller could evade the limit or poison a victim's bucket (5 spoofed posts = victim locked out). `_client_ip` now prefers Cloudflare's `cf-connecting-ip` and otherwise takes the rightmost, infrastructure-appended XFF entry (audit Medium#20). - **`CODE_OF_CONDUCT.md`** (Contributor Covenant 2.1) added and linked from `docs/contributing.md` — closes the last repo-health gap after #9642 shipped the PR template and `.editorconfig` (audit Medium#29). Also done outside this diff (session action, user-authorized): reconciled the ~50 contradictory `impl:*` labels on issue #1010 down to the 32 that match the filesystem state (8 `done`, 7 `failed`, watchdog history kept) so babysit-pipeline reads true state. ## Test plan - [x] `uv run ruff check .` + `ruff format --check .` clean (148 files) - [x] `uv run --extra typecheck mypy api core` clean - [x] `uv run --extra test pytest tests/unit tests/integration` — 1650 passed - [x] New tests: PG filters compile to `specs.tags @> …` (no CAST on the column), SQLite fallback matches literally with `%`/`_` in tag values, `get_ids_with_code` excludes NULL-code impls, `get_by_library_with_code` scopes to one library, `_client_ip` trust-order matrix (cf-connecting-ip > rightmost XFF > client.host) - [ ] PostgreSQL query plan not verified against prod (no DB in sandbox); after deploy, `EXPLAIN` an MCP tag search should show a Bitmap Index Scan on `ix_specs_tags` ## Checklist - [x] `CHANGELOG.md` updated under `[Unreleased]` (Keep-a-Changelog categories, bold-titled bullets; PR ref to be appended) - [x] Related documentation updated (`docs/contributing.md` links the new Code of Conduct) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_015kWZJgpARSK68RtfeHMEsK --- _Generated by [Claude Code](https://claude.ai/code/session_015kWZJgpARSK68RtfeHMEsK)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
impl-generate.yml's failure counter now fails closed — a rate-limited count API call previously reported "0 prior failures" and re-dispatched forever (issue [heatmap-calendar] Basic Calendar Heatmap #1010, ~1,200 runs in 38h and still firing). Also fixes per-page--jqcounting on issues with >100 comments. Merging this PR is what stops future loop iterations — the currently in-flight runs still need a manual cancel (the session's GitHub integration lacksactions:write).agentic/audits/2026-07-15-all.mdacross CI security (heredoc expansion, issue-template trigger-label gate, composite-action SHA pins), backend correctness (async Cloud SQL path, MCP NULL-tags crash, error sanitization, cache lock/GC lifecycle, seed upsert, check-constraint migration), bot SEO (site JSON-LD, spec-hub link graph, richer homepage, InspectionTool UAs), frontend (dark-mode overlays and menus, a11y pass, palette drift, context memoization), nginx security headers, node:22 build image, and a full 9/11→15-library documentation reconciliation.core/palette.pyplus new tests for every behavioral backend change; repo health: PR template +.editorconfig.Not addressable in-repo (owner action needed): Cloudflare "block AI bots" zone setting (contradicts llms.txt — documented in
app/public/robots.txt),PAGESPEED_API_KEYprovisioning, Dependabot backlog merges, issue triage, and catalog regen for the 6 majority-rejected specs.Test plan
uv run ruff check .+ruff format --check .clean (148 files)uv run --extra typecheck mypy api corecleanuv run pytest tests/unit— 1571 passedyarn lint,yarn type-check,yarn test(609 passed),yarn build— all clean$())--sqlrendering both directions;alembic headssingle headcurl -A "Google-InspectionTool" https://anyplot.ai/Checklist
CHANGELOG.mdupdated under[Unreleased](Keep-a-Changelog categories, bold-titled bullets; PR ref appended after PR creation)🤖 Generated with Claude Code
https://claude.ai/code/session_01NVLLTR8aa5VHrx1qqqmHRY
Generated by Claude Code