Audit role archetype coverage#92
Merged
Merged
Conversation
|
Warning Review limit reached
Next review available in: 31 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
added 2 commits
July 2, 2026 22:05
Adds a comprehensive audit of the role-archetype registries, plus the
small docs-only corrections called out in the audit.
Audited:
- packages/core/src/archetypes/index.ts (7 archetypes, RoleArchetype)
- packages/intelligence/src/archetypes/ (3 archetypes, Archetype)
- packages/schemas/src/archetype.ts (Zod Archetype schema)
- packages/core/src/evaluator/index.ts (live evaluator)
- packages/cli/src/cli.ts (CLI archetypes list)
- apps/web-ui/src/app/{results,components}/* (web UI consumer)
- packages/eval/src/__tests__/fixtures.test.ts (eval fixtures)
- issues, scripts/create-issues.sh, docs/ISSUES_SEED.md, ROADMAP
equivalents in README/docs, V1_SCOPE, PHASE-1, ARCHITECTURE
Key findings:
1. Two parallel archetype registries (core vs intelligence) on
divergent schemas. CLI + Web UI use core (7); prompts + eval
fixtures use intelligence (3).
2. README claimed 8 archetypes; core has 7, intelligence has 3.
Fixed to 7 in README, MVP_RELEASE_STATUS, PR_CLEANUP_HANDOFF.
3. Plain Product Manager detection is broken in the live system:
core has only ai-product-manager, so a non-AI PM CV falls back
to backend-engineer.
Files changed:
- docs/ARCHETYPE_GAP_AUDIT.md (new, 200+ lines)
- README.md: '8 role archetypes' -> '7', removed Machine Learning
Engineer from 'currently built-in' list (not in core), added
pointer to the audit
- docs/MVP_RELEASE_STATUS.md: 8 archetypes -> 7 in two places
- docs/PR_CLEANUP_HANDOFF.md: clarified 3 archetypes in intelligence
vs 7 in core
The audit draft was written before the GitHub API assigned the actual issue numbers. It referenced 'issue #97' in three places for the plain-Product-Manager follow-up. The real issue number is #90 (verified via gh issue view). - Line 135: 'issue #97, see footer' -> 'issue #90, see footer' - Line 194: 'Filed as issue #97' -> 'Filed as issue #90' - Line 204: 'follow-up to #97' -> 'follow-up to #90' Also caught by this rebase onto latest main (docs: align repository documentation with MVP status, PR #88). The rebase was conflict-free because PR #88 and the audit modified orthogonal sections of the shared files (README.md, docs/MVP_RELEASE_STATUS.md, and docs/PR_CLEANUP_HANDOFF.md). The rebase confirmed that all five audit findings survived intact: - core registry has 7 live archetypes - intelligence registry has 3 archetypes - plain Product Manager is missing from core - #90 tracks plain Product Manager - #91 tracks registry consolidation No code changes. Docs only. PR #37 and PR #78 untouched. Validates: pnpm test 12/12, pnpm lint 0 errors, pnpm build 6/6.
SaharPak
force-pushed
the
docs/archetype-gap-audit
branch
from
237944b to
4285975
Compare
SaharPak
pushed a commit
that referenced
this pull request
Jul 2, 2026
- Update snapshot commit hash: 4e43a3c -> 4b4bb03 (post-#88 + #92) - Total merged PRs: 26 -> 28 (#88 and #92 both merged by SaharPak) - Distinct merged-PR authors: 10 (unchanged) - SaharPak merged-PR count: 6 -> 8 - Open PRs: remove #88 (now merged), add #89 (this PR) - Notable contribution areas: add 'Docs consistency audit' and 'Archetype gap audit' - Privacy: drop private-email field from anonymous-contributor refresh recipe - Last refreshed footer: note PR #88 and PR #92 - Maintainers + Docs sections: credit PR #92 archetype gap audit - Archetype count: 8 -> 7 in IKetutWidiyane and Mohammadreza-Sarvari sections
SaharPak
added a commit
that referenced
this pull request
Jul 2, 2026
* docs: add contributor recognition * docs: tighten contributor recognition accuracy and privacy - Fix distinct merged-PR author count: 9 -> 10 (verified via gh pr list) - Add Cleanup Bot to the Anonymous commits on main table (22 commits) - Add explicit caveat: counts do not capture review, discussion, product thinking, or community support - Remove private email a.hakimi@mohaymen.ir from caveat 4 - Remove fabricated github.com/a.hakimi link (returns 404) - Soften 'not a real contributor' to 'not a human contributor' - Rename 'Top contributors by merged-PR count' to 'Contributors by merged-PR count' to reduce ranking feel * docs: refresh contributor recognition after PR #88 and #92 - Update snapshot commit hash: 4e43a3c -> 4b4bb03 (post-#88 + #92) - Total merged PRs: 26 -> 28 (#88 and #92 both merged by SaharPak) - Distinct merged-PR authors: 10 (unchanged) - SaharPak merged-PR count: 6 -> 8 - Open PRs: remove #88 (now merged), add #89 (this PR) - Notable contribution areas: add 'Docs consistency audit' and 'Archetype gap audit' - Privacy: drop private-email field from anonymous-contributor refresh recipe - Last refreshed footer: note PR #88 and PR #92 - Maintainers + Docs sections: credit PR #92 archetype gap audit - Archetype count: 8 -> 7 in IKetutWidiyane and Mohammadreza-Sarvari sections * docs: fix contributor formatting command --------- Co-authored-by: Cleanup Bot <cleanup-bot@example.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a comprehensive audit of the project's role archetype registries
plus the small docs-only corrections the audit calls out. No code
changes to either registry are made by this PR; two follow-up issues
are opened instead.
What was audited
packages/core/src/archetypes/index.ts(7 archetypes,RoleArchetype)packages/intelligence/src/archetypes/index.tsplus the threearchetype files it imports (3 archetypes, Zod
Archetype)packages/schemas/src/archetype.ts(source of the Zod schema)packages/core/src/types.ts(RoleArchetype)core.detectArchetype,intelligence.detectArchetype)packages/core/src/evaluator/index.ts) andits consumers:
packages/cli/src/cli.ts(thearchetypesCLI command)apps/web-ui/src/app/results/page.tsx(the Web UI results page)apps/web-ui/src/app/components/EvaluateForm.tsxpackages/eval/src/__tests__/fixtures.test.ts)and the three golden fixtures (
swe-strong,pm-strong,data-ml-strong)docs/V1_SCOPE.md,docs/PHASE-1.md,docs/ARCHITECTURE.mdscripts/create-issues.shanddocs/ISSUES_SEED.mdgh issue list --label archetypedocs/MVP_RELEASE_STATUS.md) andthe cleanup handoff (
docs/PR_CLEANUP_HANDOFF.md)Current implemented archetypes (live, CLI + Web UI)
The
coreregistry (the only one the user-facing evaluationsurface sees):
The
intelligenceregistry (used by prompts + eval fixtures):The two registries use different types (
RoleArchetypevs ZodArchetype), different IDs (ai-product-managervsproduct-manager, nosoftware-engineerin core, etc.), andno shared source of truth. Detailed mapping with which
archetype is in which registry is in the audit document.
Missing / high-priority archetypes
Highest priority (live impact, small scope): Plain
Product Manageris missing fromcore. The live detectorfalls back to
backend-engineerfor non-AI PM CVs, producinga misleading "Detected archetype: Backend Engineer" on what is
actually a PM resume. Filed as follow-up issue #90.
Architectural (medium-to-large refactor, recommend a dedicated
PR): Duplicate
corevsintelligenceregistry on divergentschemas. Until merged, every new archetype has to be added in
two places and stays at risk of drift. Filed as follow-up issue
#91.
Already open (no new issues created for these):
ai-engineer)ai-product-manager)These already have issue templates ready and are good
good first issuecandidates for community contributors.Issues created
packages/core(no AI keywords). Closes the highest-priority detection gap.
corevs
intelligence). Architectural follow-up.What was intentionally NOT implemented
archetypes from either registry.
RoleArchetype(plain TS) and theZod
Archetypeare left as-is; that's the work for thefollow-up PR on issue Consolidate the duplicate archetype registries (core vs intelligence) #91.
apps/web-uiorpackages/cli. Verified theirconsume-paths; not changed.
docs-and-issue work.
Docs / consistency corrections (in this PR)
README claimed "8 role archetypes" and listed
Machine Learning Engineeras currently built-in. The livecoreregistry has 7archetypes and no
ml-engineer(it hasai-engineerinstead).This PR corrects:
README.md— "8 role archetypes" → "7 role archetypes",removed
Machine Learning Engineerline, added a pointer tothe new audit.
docs/MVP_RELEASE_STATUS.md— "8 archetypes" → "7 archetypes"in two places, added a pointer to the audit.
docs/PR_CLEANUP_HANDOFF.md— "rubric v1 + 8 archetypes" →"rubric v1 + 3 archetypes (used by prompts + eval fixtures)
with a note pointing to the audit and clarifying the 7 in core.
Validation results
pnpm lintinfonote onbiome.jsonis unchanged frommain.pnpm testpnpm buildConfirmation that PR #37 and PR #78 were not touched
riddhij-7— CLI color output): not touched.Status: author rebase still needed; the open PR and its
contributor are unchanged.
Isusami— Cloudflare Pages): not touched.Status: awaiting
CLOUDFLARE_API_TOKEN+CLOUDFLARE_ACCOUNT_IDprovisioning; unchanged.Neither PR was checked out, rebased, edited, or commented on by
this work.