nexus: add PymtHouse capability integration follow-ups#307
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds PymtHouse billing integration: env/config, device-flow middleware/UI, token and usage APIs, discovery provider scoping with manifest sync and caching, developer key expiry, discovery service client, UI/Docs/tests updates, and plugin CSS lifecycle. ChangesEnd-to-end PymtHouse integration and discovery updates
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
|
🗃️ Database Migration Detected This PR includes changes to Prisma schema files. Please ensure:
Requesting review from the core team: @livepeer/core |
|
|
fc19916 to
3a03531
Compare
3a03531 to
47b9fd4
Compare
c75c271 to
8bbc282
Compare
…support Wire `upstreamStaticBody` through the connector template pipeline (loader interface, JSON schema, admin template route, seed script) so endpoints can send a pre-configured request body to the upstream service. Add the `clickhouse-query` connector template with four endpoints: - /network_prices — static SQL for orchestrator pricing data - /query — dynamic SELECT-only queries with regex + blacklist - /ping — ClickHouse health check - /tables — list tables via SHOW TABLES Include a how-to guide with dashboard visualization example and update the connector catalog. Made-with: Cursor
- Use ?? null instead of || null for upstreamStaticBody to preserve empty strings and match the seed script's nullish coalescing - Remove hardcoded allowedHosts from clickhouse-query.json so host validation derives from upstreamBaseUrl at runtime, avoiding conflicts when users override the base URL - Remove envKey from clickhouse-query.json — a single env var cannot map to two separate Basic-auth secrets (username + password); users configure credentials via the Settings tab UI instead - Add JSDoc docstrings to all exported interfaces and functions in the connector template loader and route handlers Made-with: Cursor
…E tests - Remove NEXT_PUBLIC_APP_URL/NEXTAUTH_URL from committed .env (use .env.local) - Patch start.sh to sync NEXT_PUBLIC_APP_URL with SHELL_PORT on every start - Remove hardcoded localhost:3000 from CORS allowlist in next.config.js - Consolidate 7 inline localhost fallbacks to use shared appUrl from lib/env - Fix layout.tsx stale localhost:3001 fallback - Use request.url origin for ClickHouse gateway URL resolution - Add auth headers + credentials to plugin frontend API calls - Make Playwright webServer.url respect PLAYWRIGHT_BASE_URL - Add orchestrator-leaderboard E2E spec (stub + live modes) - Add resolveClickhouseGatewayQueryUrl unit tests Made-with: Cursor
…d webhook guide - Add TabNav component for Leaderboard / Plans tab switching - Add PlansOverviewPage with summary stats, plan cards, endpoint info - Add PlanDetailPage with live-editing config (SLA weights, min score, topN, sort, filters) and "Apply Changes" to see results update - Add EndpointGuide component showing API endpoint URL, copy button, and 3-step webhook setup guide for signer ORCHESTRATOR_DISCOVERY_URL - Add usePlans/usePlanDetail hooks for data fetching and state management - Add frontend API functions: fetchPlans, fetchPlanResults, updatePlan, seedDemoPlans - Add backend POST /plans/seed route for dev-only demo data (4 plans) Made-with: Cursor
Migrate orchestrator-leaderboard plugin from hardcoded Tailwind gray/blue classes to NaaP's CSS-variable-based design tokens. Rewrites tailwind.config.js with semantic color mappings, globals.css with :root/.dark CSS variables and .glass-card, and all 7 TSX component files with token-backed utility classes. Also fixes a bug where updating a plan's configuration (e.g. SLA min score) did not refresh the orchestrator results — the PUT handler now invalidates the in-memory planCache entry so the next GET /results forces a fresh evaluation. Made-with: Cursor
Made-with: Cursor
… scoping
- Fix auth redirect loop: always clear httpOnly cookie via logout endpoint
before redirecting to /login, preventing middleware loop when DB is
unreachable or session is invalid
- Fix plan visibility: use OR logic in scopeWhere so plans match by either
teamId or ownerUserId (build-time seed sets ownerUserId only)
- Fix seed button hidden on Vercel: remove isLocalhost gate so all
authenticated users can seed demo data
- Fix seed API blocked on Vercel: remove NODE_ENV=production check that
blocked preview deployments
- Fix per-user seeding: use user-scoped billingPlanIds (demo-{userId}-slug)
so each user gets their own isolated set of demo plans without unique
constraint collisions
- Set teamId in build-time seed for proper scope matching
Made-with: Cursor
…ct resolver and audit log Replace the hard-coded ClickHouse-only refresh pipeline with a pluggable multi-source architecture: - 4 source adapters: livepeer-subgraph (on-chain registry), clickhouse-query, naap-discover, naap-pricing — each behind the SourceAdapter interface - 3 new Service Gateway connector templates for the new upstream APIs - Hybrid conflict resolver: source-level membership + field-level metric priority with ethAddress↔orchUri cross-reference joining - Durable audit log (LeaderboardRefreshAudit) recording per-source stats, conflicts, dropped rows, and warnings on every refresh - Admin-configurable source priority/toggle via LeaderboardSource Prisma model - GET/PUT /sources and GET /audits REST endpoints with admin authz - Admin UI: tabbed settings with drag-to-reorder Data Sources panel and expandable Refresh Audit log - Comprehensive docs: data-sources.md, updated openapi.yaml, api-reference.md, how-to-guide.md, for-ai.md - 20 new unit tests (resolver + adapters) + E2E test spec - All 119 existing tests continue to pass Co-authored-by: Cursor <cursoragent@cursor.com>
- Introduced a new script command `seed:discovery-plans` in package.json to facilitate the seeding of discovery plans, enhancing the setup process for development and testing environments.
…st handling - Updated the orchestrator leaderboard API to support a new `manifestOnly` query parameter, allowing for streamlined responses when only the manifest is requested. - Replaced the deprecated `syncPymthouseManifestSnapshot` function with `ensurePymthouseManifestFresh` to improve manifest freshness checks. - Enhanced caching strategies by setting appropriate `Cache-Control` headers for responses, ensuring better performance and data consistency. - Refactored the `getDashboardPipelineCatalog` function to accept an optional `bypassCache` parameter, allowing for more flexible data retrieval. - Improved capability filtering logic to respect provider restrictions, ensuring only allowed capabilities are returned based on the billing provider. - Updated documentation to reflect changes in API behavior and caching mechanisms.
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
apps/web-next/src/lib/orchestrator-leaderboard/refresh.ts (1)
188-199:⚠️ Potential issue | 🟡 Minor | ⚡ Quick win
getCachedPlanResultsreturns the first matching entry, which may not be the freshest.When multiple composite cache entries exist for the same
planId(e.g., after capability changes), this returns the first valid one found during iteration. Map iteration order is insertion order, so this could return a stale entry if a newer one was inserted later. Consider returning the entry with the most recentcachedAttimestamp.Proposed fix
export function getCachedPlanResults(planId: string): PlanResults | null { const prefix = `${planId}${PLAN_CACHE_KEY_SEP}`; + let best: PlanCacheEntry | null = null; for (const [key, entry] of planCache) { if (!key.startsWith(prefix)) continue; if (!isValid(entry)) continue; - return { - ...entry.results, - meta: { ...entry.results.meta, cacheAgeMs: Date.now() - entry.cachedAt }, - }; + if (!best || entry.cachedAt > best.cachedAt) { + best = entry; + } } - return null; + if (!best) return null; + return { + ...best.results, + meta: { ...best.results.meta, cacheAgeMs: Date.now() - best.cachedAt }, + }; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/web-next/src/lib/orchestrator-leaderboard/refresh.ts` around lines 188 - 199, getCachedPlanResults currently iterates planCache and returns the first valid entry for a given planId which can be stale due to insertion order; update getCachedPlanResults to scan all entries whose keys start with `${planId}${PLAN_CACHE_KEY_SEP}`, filter by isValid(entry), and select the entry with the largest entry.cachedAt (newest) before returning its results (still adding cacheAgeMs as Date.now() - cachedAt); use the existing symbols planCache, PLAN_CACHE_KEY_SEP, isValid, cachedAt, and PlanResults to locate and implement this change.plugins/orchestrator-leaderboard/frontend/src/pages/PlanCreatePage.tsx (1)
164-172:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winClamp
topNbefore state update.Current parsing can still pass invalid values (e.g., negative or >1000) into request payloads. Clamp in code, not only via input attributes.
Suggested fix
<input type="number" min={1} max={1000} value={topN} - onChange={(e) => setTopN(Number(e.target.value) || 10)} + onChange={(e) => { + const n = Number(e.target.value); + if (!Number.isFinite(n)) { + setTopN(10); + return; + } + setTopN(Math.min(1000, Math.max(1, Math.trunc(n)))); + }} className="w-full px-3 py-2 bg-bg-secondary border border-[var(--border-color)] rounded-lg text-text-primary text-sm" />🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/orchestrator-leaderboard/frontend/src/pages/PlanCreatePage.tsx` around lines 164 - 172, The Top N input currently uses onChange={(e) => setTopN(Number(e.target.value) || 10)} which can still allow values outside [1,1000] and mishandle 0; update the onChange handler in PlanCreatePage (the code that reads topN and calls setTopN) to parse the value, handle NaN by falling back to a sane default, and clamp it with Math.max(1, Math.min(1000, parsedValue)) before calling setTopN; also ensure any place that reads topN for request payloads uses the clamped value (e.g., before submit) so only valid values are sent.
🧹 Nitpick comments (4)
apps/web-next/src/lib/orchestrator-leaderboard/refresh.ts (2)
220-228: 💤 Low value
startLocalRefreshLoopignores passedauthToken.The function accepts
authTokenas a parameter but never forwards it torefreshAllPlans(). Either pass the token or remove the unused parameter.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/web-next/src/lib/orchestrator-leaderboard/refresh.ts` around lines 220 - 228, startLocalRefreshLoop currently accepts authToken but never uses it; update the setInterval callback to forward the token to refreshAllPlans (e.g. refreshAllPlans(authToken).catch(...)) so the auth context is preserved, or if the token is not needed remove the authToken parameter from startLocalRefreshLoop; locate the function startLocalRefreshLoop and the call to refreshAllPlans to apply the change.
105-109: 💤 Low valueUnused parameters in
evaluateAndCache.The parameters
authToken,requestUrl, andcookieHeaderare declared but never used in the function body. If these are placeholders for future functionality, consider documenting that intent or removing them to avoid confusion.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/web-next/src/lib/orchestrator-leaderboard/refresh.ts` around lines 105 - 109, The function evaluateAndCache currently declares unused parameters authToken, requestUrl, and cookieHeader; either remove these parameters from the evaluateAndCache signature (and update any call sites) to eliminate dead parameters, or if they are intentionally reserved for future use, rename them to _authToken, _requestUrl, and _cookieHeader (or add a short JSDoc/TODO noting they are intentionally unused) so linters and readers know they are placeholders; locate the symbol evaluateAndCache to apply the change and ensure consistency across callers.bin/db-setup.sh (1)
86-88: 💤 Low value
A && B || Cpattern may run C if B fails (SC2015).While unlikely since
log_successis just an echo, the static analysis warning is technically valid. The warning also applies to line 83. For robustness, consider using explicit if-then-else, though this is low risk given the simple logging functions.Safer alternative
-npx tsx bin/seed-discovery-plans.ts 2>/dev/null && log_success "Default discovery plans seeded" || log_warn "Discovery plan seed had issues (non-critical)" +if npx tsx bin/seed-discovery-plans.ts 2>/dev/null; then + log_success "Default discovery plans seeded" +else + log_warn "Discovery plan seed had issues (non-critical)" +fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@bin/db-setup.sh` around lines 86 - 88, The current use of the A && B || C pattern (running npx tsx ... && log_success ... || log_warn ...) can call the fallback even if the success logging command fails (shellcheck SC2015); update the seed commands (referencing log_info, log_success, log_warn and the npx tsx bin/seed-discovery-plans.ts invocation executed from ROOT_DIR) to use an explicit if-then-else: run the npx command and check its exit status with if ...; then call log_success; else call log_warn; fi (apply the same change to the similar pattern earlier on line with the other seed command too).apps/web-next/src/lib/pymthouse-manifest.ts (1)
56-61: 💤 Low valueEnvironment variable fallbacks include typo variants.
The code falls back from
PYMTHOUSE_*toPMTHOUSE_*(missing 'Y'). If this is intentional for backwards compatibility, consider adding a deprecation warning when the typo variant is used. Otherwise, this may silently accept misconfigured environments.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/web-next/src/lib/pymthouse-manifest.ts` around lines 56 - 61, The fallback uses misspelled env names (PMTHOUSE_*) for publicId, m2mId and m2mSecret; either remove the typo fallbacks to require the correct PYMTHOUSE_* vars, or keep them but detect when the PMTHOUSE_* variant is used and emit a deprecation warning (e.g., console.warn or existing logger) indicating the correct PYMTHOUSE_* variable to use; update the logic around publicId, m2mId and m2mSecret to check which source was used and log the deprecation when PMTHOUSE_* is present.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@apps/web-next/src/app/api/v1/orchestrator-leaderboard/python-gateway/route.ts`:
- Around line 87-89: The call to ensurePymthouseManifestFresh() is placed
outside the existing try/catch so any errors bypass your route's error boundary;
move the conditional block that calls ensurePymthouseManifestFresh() (the if
(billingProvider === 'pymthouse') { await ensurePymthouseManifestFresh(); }
logic) inside the same try that wraps the route processing so failures are
caught by the route's catch and correct status mapping is applied.
In `@plugins/orchestrator-leaderboard/frontend/src/components/StyledCheckbox.tsx`:
- Around line 21-33: The visible checkbox isn’t clickable because the native
<input> is sr-only and not associated with the visible span; wrap the input and
the visible span in a clickable label (or make the input absolutely fill the
container and be invisible but not sr-only) inside the StyledCheckbox component
so clicks on the rendered box toggle the input; keep the input’s checked,
disabled, ariaLabel, onChange handler and the peer class intact (update the
input’s classes to either remove sr-only and use "absolute inset-0 w-full h-full
opacity-0 cursor-pointer" or wrap both elements in a <label> so the span remains
a visual-only element while clicks target the input).
In `@plugins/orchestrator-leaderboard/frontend/src/globals.css`:
- Around line 108-113: The Stylelint rule "declaration-empty-line-before" is
being violated around the .section-label-text and nearby selector blocks; update
the CSS so each selector's first declaration conforms to the project's rule by
either inserting a single blank line before the declaration block or removing an
unexpected blank line as required by the configured rule; specifically adjust
the .section-label-text rule and the other affected selector near line 148 so
the spacing before their first property matches the configured
declaration-empty-line-before behavior (ensure consistent blank-line
presence/absence across those selector blocks).
In `@plugins/orchestrator-leaderboard/frontend/src/pages/LeaderboardPage.tsx`:
- Around line 70-80: The auto-refresh toggle button does not expose its pressed
state to assistive tech; update the button (the element that calls
setAutoRefresh and reads autoRefresh, near Radio) to include
aria-pressed={autoRefresh} and add a stateful aria-label (e.g., "Auto-refresh
on" / "Auto-refresh off" derived from autoRefresh) so screen readers receive the
toggle semantics while preserving the existing onClick behavior and visual
classes.
---
Outside diff comments:
In `@apps/web-next/src/lib/orchestrator-leaderboard/refresh.ts`:
- Around line 188-199: getCachedPlanResults currently iterates planCache and
returns the first valid entry for a given planId which can be stale due to
insertion order; update getCachedPlanResults to scan all entries whose keys
start with `${planId}${PLAN_CACHE_KEY_SEP}`, filter by isValid(entry), and
select the entry with the largest entry.cachedAt (newest) before returning its
results (still adding cacheAgeMs as Date.now() - cachedAt); use the existing
symbols planCache, PLAN_CACHE_KEY_SEP, isValid, cachedAt, and PlanResults to
locate and implement this change.
In `@plugins/orchestrator-leaderboard/frontend/src/pages/PlanCreatePage.tsx`:
- Around line 164-172: The Top N input currently uses onChange={(e) =>
setTopN(Number(e.target.value) || 10)} which can still allow values outside
[1,1000] and mishandle 0; update the onChange handler in PlanCreatePage (the
code that reads topN and calls setTopN) to parse the value, handle NaN by
falling back to a sane default, and clamp it with Math.max(1, Math.min(1000,
parsedValue)) before calling setTopN; also ensure any place that reads topN for
request payloads uses the clamped value (e.g., before submit) so only valid
values are sent.
---
Nitpick comments:
In `@apps/web-next/src/lib/orchestrator-leaderboard/refresh.ts`:
- Around line 220-228: startLocalRefreshLoop currently accepts authToken but
never uses it; update the setInterval callback to forward the token to
refreshAllPlans (e.g. refreshAllPlans(authToken).catch(...)) so the auth context
is preserved, or if the token is not needed remove the authToken parameter from
startLocalRefreshLoop; locate the function startLocalRefreshLoop and the call to
refreshAllPlans to apply the change.
- Around line 105-109: The function evaluateAndCache currently declares unused
parameters authToken, requestUrl, and cookieHeader; either remove these
parameters from the evaluateAndCache signature (and update any call sites) to
eliminate dead parameters, or if they are intentionally reserved for future use,
rename them to _authToken, _requestUrl, and _cookieHeader (or add a short
JSDoc/TODO noting they are intentionally unused) so linters and readers know
they are placeholders; locate the symbol evaluateAndCache to apply the change
and ensure consistency across callers.
In `@apps/web-next/src/lib/pymthouse-manifest.ts`:
- Around line 56-61: The fallback uses misspelled env names (PMTHOUSE_*) for
publicId, m2mId and m2mSecret; either remove the typo fallbacks to require the
correct PYMTHOUSE_* vars, or keep them but detect when the PMTHOUSE_* variant is
used and emit a deprecation warning (e.g., console.warn or existing logger)
indicating the correct PYMTHOUSE_* variable to use; update the logic around
publicId, m2mId and m2mSecret to check which source was used and log the
deprecation when PMTHOUSE_* is present.
In `@bin/db-setup.sh`:
- Around line 86-88: The current use of the A && B || C pattern (running npx tsx
... && log_success ... || log_warn ...) can call the fallback even if the
success logging command fails (shellcheck SC2015); update the seed commands
(referencing log_info, log_success, log_warn and the npx tsx
bin/seed-discovery-plans.ts invocation executed from ROOT_DIR) to use an
explicit if-then-else: run the npx command and check its exit status with if
...; then call log_success; else call log_warn; fi (apply the same change to the
similar pattern earlier on line with the other seed command too).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: b2c520cd-f0b0-4c32-8917-7749c8bb6b89
📒 Files selected for processing (44)
apps/web-next/src/app/api/v1/orchestrator-leaderboard/capability-catalog/route.tsapps/web-next/src/app/api/v1/orchestrator-leaderboard/plans/[id]/python-gateway/route.tsapps/web-next/src/app/api/v1/orchestrator-leaderboard/plans/[id]/results/route.tsapps/web-next/src/app/api/v1/orchestrator-leaderboard/plans/seed/route.tsapps/web-next/src/app/api/v1/orchestrator-leaderboard/python-gateway/route.tsapps/web-next/src/components/layout/app-layout.tsxapps/web-next/src/components/layout/top-bar.tsxapps/web-next/src/lib/discovery-service/client.tsapps/web-next/src/lib/facade/index.tsapps/web-next/src/lib/facade/resolvers/pipeline-catalog.tsapps/web-next/src/lib/orchestrator-leaderboard/__tests__/provider-restrictions.test.tsapps/web-next/src/lib/orchestrator-leaderboard/__tests__/types-validation.test.tsapps/web-next/src/lib/orchestrator-leaderboard/refresh.tsapps/web-next/src/lib/orchestrator-leaderboard/types.tsapps/web-next/src/lib/orchestrators-discovery-policy.tsapps/web-next/src/lib/pymthouse-manifest.test.tsapps/web-next/src/lib/pymthouse-manifest.tsapps/web-next/tests/leaderboard-posture.spec.tsapps/workflows/developer-web/src/pages/DeveloperView.tsxbin/db-setup.shbin/vercel-build.shdocs/pymthouse-integration.mdpackage.jsonplugins/developer-api/frontend/src/pages/DeveloperView.tsxplugins/orchestrator-leaderboard/docs/how-to-guide.mdplugins/orchestrator-leaderboard/docs/openapi.yamlplugins/orchestrator-leaderboard/frontend/src/App.tsxplugins/orchestrator-leaderboard/frontend/src/components/AdminSettings.tsxplugins/orchestrator-leaderboard/frontend/src/components/CapabilityGroupPicker.tsxplugins/orchestrator-leaderboard/frontend/src/components/CapabilityTag.tsxplugins/orchestrator-leaderboard/frontend/src/components/CollapsibleTagList.tsxplugins/orchestrator-leaderboard/frontend/src/components/EndpointGuide.tsxplugins/orchestrator-leaderboard/frontend/src/components/FormLabel.tsxplugins/orchestrator-leaderboard/frontend/src/components/SectionLabel.tsxplugins/orchestrator-leaderboard/frontend/src/components/StyledCheckbox.tsxplugins/orchestrator-leaderboard/frontend/src/components/TabNav.tsxplugins/orchestrator-leaderboard/frontend/src/globals.cssplugins/orchestrator-leaderboard/frontend/src/hooks/useCapabilityCatalog.tsplugins/orchestrator-leaderboard/frontend/src/hooks/usePlans.tsplugins/orchestrator-leaderboard/frontend/src/lib/api.tsplugins/orchestrator-leaderboard/frontend/src/pages/LeaderboardPage.tsxplugins/orchestrator-leaderboard/frontend/src/pages/PlanCreatePage.tsxplugins/orchestrator-leaderboard/frontend/src/pages/PlanDetailPage.tsxplugins/orchestrator-leaderboard/frontend/src/pages/PlansOverviewPage.tsx
💤 Files with no reviewable changes (2)
- apps/web-next/src/app/api/v1/orchestrator-leaderboard/plans/seed/route.ts
- plugins/orchestrator-leaderboard/docs/openapi.yaml
✅ Files skipped from review due to trivial changes (6)
- apps/workflows/developer-web/src/pages/DeveloperView.tsx
- plugins/orchestrator-leaderboard/frontend/src/components/TabNav.tsx
- plugins/orchestrator-leaderboard/frontend/src/components/SectionLabel.tsx
- bin/vercel-build.sh
- package.json
- docs/pymthouse-integration.md
| if (billingProvider === 'pymthouse') { | ||
| await ensurePymthouseManifestFresh(); | ||
| } |
There was a problem hiding this comment.
Move manifest refresh inside the existing error boundary
ensurePymthouseManifestFresh() is outside the try, so any refresh failure escapes your route-level error handling and status mapping. Keep it inside the try to preserve consistent API behavior.
Proposed fix
- if (billingProvider === 'pymthouse') {
- await ensurePymthouseManifestFresh();
- }
-
try {
+ if (billingProvider === 'pymthouse') {
+ await ensurePymthouseManifestFresh();
+ }
+
const ordered: string[] = [];📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (billingProvider === 'pymthouse') { | |
| await ensurePymthouseManifestFresh(); | |
| } | |
| try { | |
| if (billingProvider === 'pymthouse') { | |
| await ensurePymthouseManifestFresh(); | |
| } | |
| const ordered: string[] = []; |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@apps/web-next/src/app/api/v1/orchestrator-leaderboard/python-gateway/route.ts`
around lines 87 - 89, The call to ensurePymthouseManifestFresh() is placed
outside the existing try/catch so any errors bypass your route's error boundary;
move the conditional block that calls ensurePymthouseManifestFresh() (the if
(billingProvider === 'pymthouse') { await ensurePymthouseManifestFresh(); }
logic) inside the same try that wraps the route processing so failures are
caught by the route's catch and correct status mapping is applied.
| <span className="relative inline-flex items-center justify-center shrink-0 w-3.5 h-3.5"> | ||
| <input | ||
| type="checkbox" | ||
| checked={checked} | ||
| disabled={disabled} | ||
| aria-label={ariaLabel} | ||
| onChange={(e) => onChange(e.target.checked)} | ||
| className="peer sr-only" | ||
| /> | ||
| <span | ||
| aria-hidden | ||
| className="pointer-events-none absolute inset-0 rounded border border-zinc-600 bg-zinc-800 transition-colors peer-checked:bg-[var(--accent-emerald)] peer-checked:border-[var(--accent-emerald)] peer-disabled:opacity-50 peer-focus-visible:ring-2 peer-focus-visible:ring-[var(--accent-emerald)] peer-focus-visible:ring-offset-1 peer-focus-visible:ring-offset-[var(--bg-secondary)]" | ||
| /> |
There was a problem hiding this comment.
Make the visible checkbox area actually clickable.
On Line 22, the native input is sr-only inside a non-label wrapper, so the visible box isn’t a reliable click target. This can make mouse interaction fail for most of the rendered control.
🔧 Proposed fix
-export const StyledCheckbox: React.FC<StyledCheckboxProps> = ({
+export const StyledCheckbox: React.FC<StyledCheckboxProps> = ({
checked,
disabled = false,
onChange,
'aria-label': ariaLabel,
}) => (
- <span className="relative inline-flex items-center justify-center shrink-0 w-3.5 h-3.5">
+ <label className="relative inline-flex items-center justify-center shrink-0 w-3.5 h-3.5 cursor-pointer">
<input
type="checkbox"
checked={checked}
disabled={disabled}
aria-label={ariaLabel}
onChange={(e) => onChange(e.target.checked)}
- className="peer sr-only"
+ className="peer absolute inset-0 h-full w-full opacity-0 cursor-pointer"
/>
<span
aria-hidden
className="pointer-events-none absolute inset-0 rounded border border-zinc-600 bg-zinc-800 transition-colors peer-checked:bg-[var(--accent-emerald)] peer-checked:border-[var(--accent-emerald)] peer-disabled:opacity-50 peer-focus-visible:ring-2 peer-focus-visible:ring-[var(--accent-emerald)] peer-focus-visible:ring-offset-1 peer-focus-visible:ring-offset-[var(--bg-secondary)]"
/>
<Check
size={9}
aria-hidden
className="relative z-[1] text-white opacity-0 transition-opacity peer-checked:opacity-100 pointer-events-none"
/>
- </span>
+ </label>
);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@plugins/orchestrator-leaderboard/frontend/src/components/StyledCheckbox.tsx`
around lines 21 - 33, The visible checkbox isn’t clickable because the native
<input> is sr-only and not associated with the visible span; wrap the input and
the visible span in a clickable label (or make the input absolutely fill the
container and be invisible but not sr-only) inside the StyledCheckbox component
so clicks on the rendered box toggle the input; keep the input’s checked,
disabled, ariaLabel, onChange handler and the peer class intact (update the
input’s classes to either remove sr-only and use "absolute inset-0 w-full h-full
opacity-0 cursor-pointer" or wrap both elements in a <label> so the span remains
a visual-only element while clicks target the input).
| <button | ||
| onClick={() => setAutoRefresh(!autoRefresh)} | ||
| className={`flex items-center gap-1.5 px-3 py-1.5 rounded-lg text-xs font-medium transition-all ${ | ||
| autoRefresh | ||
| ? 'bg-accent-emerald/15 text-accent-emerald border border-accent-emerald/30' | ||
| : 'bg-bg-secondary text-text-secondary border border-[var(--border-color)] hover:border-white/20' | ||
| }`} | ||
| > | ||
| <Radio size={12} className={autoRefresh ? 'animate-pulse' : ''} /> | ||
| {autoRefresh ? 'Live' : 'Auto-refresh'} | ||
| </button> |
There was a problem hiding this comment.
Expose toggle state semantics for the auto-refresh control.
This button behaves like a toggle, but it doesn’t currently expose pressed state for assistive tech. Add aria-pressed (and optionally a stateful aria-label).
Suggested fix
<button
onClick={() => setAutoRefresh(!autoRefresh)}
+ aria-pressed={autoRefresh}
+ aria-label={autoRefresh ? 'Disable auto-refresh' : 'Enable auto-refresh'}
className={`flex items-center gap-1.5 px-3 py-1.5 rounded-lg text-xs font-medium transition-all ${
autoRefresh
? 'bg-accent-emerald/15 text-accent-emerald border border-accent-emerald/30'
: 'bg-bg-secondary text-text-secondary border border-[var(--border-color)] hover:border-white/20'
}`}
>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| <button | |
| onClick={() => setAutoRefresh(!autoRefresh)} | |
| className={`flex items-center gap-1.5 px-3 py-1.5 rounded-lg text-xs font-medium transition-all ${ | |
| autoRefresh | |
| ? 'bg-accent-emerald/15 text-accent-emerald border border-accent-emerald/30' | |
| : 'bg-bg-secondary text-text-secondary border border-[var(--border-color)] hover:border-white/20' | |
| }`} | |
| > | |
| <Radio size={12} className={autoRefresh ? 'animate-pulse' : ''} /> | |
| {autoRefresh ? 'Live' : 'Auto-refresh'} | |
| </button> | |
| <button | |
| onClick={() => setAutoRefresh(!autoRefresh)} | |
| aria-pressed={autoRefresh} | |
| aria-label={autoRefresh ? 'Disable auto-refresh' : 'Enable auto-refresh'} | |
| className={`flex items-center gap-1.5 px-3 py-1.5 rounded-lg text-xs font-medium transition-all ${ | |
| autoRefresh | |
| ? 'bg-accent-emerald/15 text-accent-emerald border border-accent-emerald/30' | |
| : 'bg-bg-secondary text-text-secondary border border-[var(--border-color)] hover:border-white/20' | |
| }`} | |
| > | |
| <Radio size={12} className={autoRefresh ? 'animate-pulse' : ''} /> | |
| {autoRefresh ? 'Live' : 'Auto-refresh'} | |
| </button> |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@plugins/orchestrator-leaderboard/frontend/src/pages/LeaderboardPage.tsx`
around lines 70 - 80, The auto-refresh toggle button does not expose its pressed
state to assistive tech; update the button (the element that calls
setAutoRefresh and reads autoRefresh, near Radio) to include
aria-pressed={autoRefresh} and add a stateful aria-label (e.g., "Auto-refresh
on" / "Auto-refresh off" derived from autoRefresh) so screen readers receive the
toggle semantics while preserving the existing onClick behavior and visual
classes.
…nctionality - Added a modal for displaying test results and errors when running tests against the API endpoint. - Implemented a new `testPlanResultsEndpoint` function to facilitate testing of the API with the current session token. - Updated the `EndpointGuide` component to include a button for running tests, showing loading states and handling responses. - Refactored the `CollapsibleTagList` component to prevent event propagation on toggle button clicks, improving user interaction. - Renamed the section label in `PlanDetailPage` to "Discovery API" for better clarity.
- Deleted the following API routes: `/api/v1/pymthouse/capabilities/catalog`, `/api/v1/pymthouse/network/price`, and `/api/v1/pymthouse/sla/summary`. - Removed the `pymthouse-plan-builder.ts` file, which contained logic for building capabilities catalog, SLA summary, and network price payloads. - Updated documentation to reflect the removal of these routes and the new data retrieval methods from the NaaP deployment.
…er-sdk - Replaced all instances of `@pymthouse/builder-api` with `@pymthouse/builder-sdk` across the codebase. - Updated the package version in `package.json` and `package-lock.json` to `0.0.8`. - Modified environment variable documentation to reflect the new SDK usage. - Ensured all related tests and configurations are aligned with the new package structure.
- Added `rimraf` as a dependency for improved build script cleanup. - Updated `.env.local.example` to correct the environment variable name for `PYMTHOUSE_BASE_URL`. - Refactored cookie handling functions to support asynchronous operations, ensuring proper signing and parsing of device approval cookies. - Enhanced rate limiting logic to handle anonymous requests more effectively. - Improved error handling in API routes for better debugging and user feedback. - Updated tests to reflect changes in cookie handling and ensure robust functionality.
- Added new development dependencies: `cac`, `check-error`, `deep-eql`, `loupe`, `pathval`, `strip-literal`, `tinypool`, and `tinyspy` with their respective versions and metadata. - Updated `vite-node` to version `3.2.4` and included its dependencies. - Regenerated `package-lock.json` to reflect the latest changes in the dependency tree.
…mantics
After the fail-closed change to isPipelineModelInManifest (a missing
manifest now denies by default), two tests started relying on side
effects of the old fail-open behavior:
- orchestrators-discovery-policy "passes when pair is on allowlist":
the mock Response lacked `headers`, so `res.headers.get('etag')`
threw and the catch left the snapshot null. Add `headers: new
Headers()` so the manifest is actually parsed and the allow path is
exercised.
- orchestrator-leaderboard refresh.test: the mockPlan used
`billingProviderSlug: 'pymthouse'`, which triggers manifest
filtering on every evaluation. Without creds/seed the manifest is
null and every capability is filtered out. Use `null` so the cache
and ranking logic (what these tests are actually covering) runs
unaffected by manifest gating. Also add the now-required
`visibility` field.
Co-authored-by: Cursor <cursoragent@cursor.com>
Move orchestrator-leaderboard backend, manifest gating, python-gateway routes, discovery-service client, plugin UI, and docs from #307 into this PR. Excludes developer-api and billing OIDC/auth changes. Includes capability catalog, tiered discovery shuffle, default/plan-scoped python-gateway, PymtHouse manifest sync, and removal of demo seed API. Co-authored-by: Cursor <cursoragent@cursor.com>
Stack remaining #307 work on feat/discovery-plan-billing-provider: - PymtHouse OIDC device-flow login and device approval routes - Billing token/usage APIs and usage helpers - Developer API key expiry and billing OAuth hardening - developer-api plugin UI (usage, keys, discovery plan tooltip) - UMD plugin stylesheet lifecycle, dev-api packages, builder-sdk - Prisma pkceCodeVerifier on BillingProviderOAuthSession Co-authored-by: Cursor <cursoragent@cursor.com>
Stack remaining #307 work on feat/discovery-plan-billing-provider: - PymtHouse OIDC device-flow login and device approval routes - Billing token/usage APIs and usage helpers - Developer API key expiry and billing OAuth hardening - developer-api plugin UI (usage, keys, discovery plan tooltip) - UMD plugin stylesheet lifecycle, dev-api packages, builder-sdk - Prisma pkceCodeVerifier on BillingProviderOAuthSession Co-authored-by: Cursor <cursoragent@cursor.com>
Stack remaining #307 work on feat/discovery-plan-billing-provider: - PymtHouse OIDC device-flow login and device approval routes - Billing token/usage APIs and usage helpers - Developer API key expiry and billing OAuth hardening - developer-api plugin UI (usage, keys, discovery plan tooltip) - UMD plugin stylesheet lifecycle, dev-api packages, builder-sdk - Prisma pkceCodeVerifier on BillingProviderOAuthSession Co-authored-by: Cursor <cursoragent@cursor.com>
Stack remaining #307 work on feat/discovery-plan-billing-provider: - PymtHouse OIDC device-flow login and device approval routes - Billing token/usage APIs and usage helpers - Developer API key expiry and billing OAuth hardening - developer-api plugin UI (usage, keys, discovery plan tooltip) - UMD plugin stylesheet lifecycle, dev-api packages, builder-sdk - Prisma pkceCodeVerifier on BillingProviderOAuthSession Co-authored-by: Cursor <cursoragent@cursor.com>
Summary
user_codeconfirmation before approve)pymthousebilling providerpymthousewhen configuredOut of scope (follow-up PRs)
GET /api/v1/billing/pymthouse/usage+ DeveloperView metrics)DISCOVERY_SERVICE_URL)Test plan
pymthouse-device-initiate,provider-restrictions,python-gatewayroute unit testsuser_codeon consent screen → tokenSummary by CodeRabbit
New Features
Improvements