feat(assessors): add lint suppression density assessor#518
Open
msu8 wants to merge 1 commit into
Open
Conversation
|
Warning Review limit reached
Next review available in: 4 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Enterprise Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (9)
📝 WalkthroughWalkthroughAdds a configurable ChangesLint suppression density
Sequence Diagram(s)sequenceDiagram
participant Repository
participant LintSuppressionAssessor
participant Config
participant Finding
Repository->>LintSuppressionAssessor: provide repository
LintSuppressionAssessor->>Config: read thresholds and exclusions
LintSuppressionAssessor->>Repository: scan tracked source files
LintSuppressionAssessor->>LintSuppressionAssessor: count suppressions and calculate density
LintSuppressionAssessor->>Finding: return score, evidence, and remediation
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
msu8
force-pushed
the
510
branch
2 times, most recently
from
July 16, 2026 14:25
f32905b to
3962946
Compare
Contributor
📈 Test Coverage Report
Coverage calculated from unit tests only |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/attributes.md`:
- Line 6: Update the attribute counts in docs/attributes.md to match the weights
catalog: report 30 total attributes, 10 Tier 2 attributes, and 9 Tier 3
attributes. Adjust the affected summary and section text while leaving the
attribute definitions unchanged.
In `@src/agentready/assessors/code_quality.py`:
- Around line 2149-2161: Update the suppression assessor flow around
_count_file_suppressions to retain each file’s line count, rank file_stats by
suppressions divided by LOC rather than absolute counts, and use
calculate_proportional_score() for proportional scoring. Replace direct Finding
construction and manual score interpolation with Finding.create_pass() or
Finding.create_fail() while preserving the assessor’s existing pass/fail
criteria and output details.
- Around line 1943-1950: Update the JavaScript and TypeScript rule patterns in
the code-quality assessor to detect canonical block-form /* eslint-disable */
directives alongside existing line comments, while retaining `@ts-ignore`
detection. Add a regression test covering block-form ESLint suppression and
confirming it is counted.
In `@src/agentready/data/default-weights.yaml`:
- Around line 76-81: Update
CyclomaticComplexityAssessor.attribute.default_weight and
ArchitecturalBoundaryAssessor.attribute.default_weight from 0.02 to 0.01 so
assessor fallback values match the corresponding entries in
default-weights.yaml.
In `@src/agentready/models/config.py`:
- Around line 112-118: Add a typed LintSuppressionOptions model near the
configuration models and change assessor_options to use it instead of
unvalidated Any dictionaries. Configure strict boolean fields, finite
non-negative numeric thresholds, and validation requiring fail_per_kloc to
exceed pass_per_kloc, so invalid assessor settings are rejected during
configuration parsing.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 007af8eb-0b73-4298-aec6-3d28976b37ae
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (8)
.agentready-config.example.yamldocs/attributes.mdsrc/agentready/assessors/__init__.pysrc/agentready/assessors/code_quality.pysrc/agentready/data/.agentready-config.example.yamlsrc/agentready/data/default-weights.yamlsrc/agentready/models/config.pytests/unit/test_assessors_code_quality.py
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/agentready/assessors/code_quality.py`:
- Around line 2119-2127: Update the traversal loop around os.walk in the
relevant assessor to use the repository’s tracked, ignore-aware file inventory
instead of scanning every directory. Preserve the existing
_SUPPRESSION_EXCLUDED_DIRS filtering and filename matching for bare_names and
dot_exts, while retaining explicit generated/vendor exclusions and yielding the
same abs_path, rel_path pairs.
- Around line 2135-2140: Update the suppression-counting logic around the
lines-processing function to ignore directive text inside string literals,
including test fixtures and statements such as print calls. Tokenize comment
content or mask string literals before applying patterns, while preserving
counting of actual suppression directives in comments and the existing return
values.
- Around line 2096-2103: Update the JavaScript/TypeScript branch in the
test-file detection logic to recognize both JSX and TSX test/spec suffixes in
addition to JS and TS. Ensure exclude_tests correctly filters Component.test.jsx
and Component.spec.tsx while preserving the existing directory and root-prefix
checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: f728a4b2-731f-461b-b068-6702d5837e7c
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (8)
.agentready-config.example.yamldocs/attributes.mdsrc/agentready/assessors/__init__.pysrc/agentready/assessors/code_quality.pysrc/agentready/data/.agentready-config.example.yamlsrc/agentready/data/default-weights.yamlsrc/agentready/models/config.pytests/unit/test_assessors_code_quality.py
msu8
force-pushed
the
510
branch
4 times, most recently
from
July 17, 2026 06:45
5f9c4bb to
476098c
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
src/agentready/assessors/code_quality.py (2)
2080-2087: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winExclude JSX and TSX test files.
Component.test.jsxandComponent.spec.tsxare scanned even whenexclude_tests=True. Derive suffixes from_LANG_EXTENSIONS[lang]and add regressions for both forms.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/agentready/assessors/code_quality.py` around lines 2080 - 2087, Update the test-file detection logic in the JavaScript/TypeScript branch to derive test suffixes from _LANG_EXTENSIONS[lang], covering JSX and TSX alongside JS and TS. Preserve the existing directory and root-prefix checks, and add regression coverage confirming Component.test.jsx and Component.spec.tsx are excluded when exclude_tests=True.
2159-2173: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winRank files by suppression concentration, not count.
A large file with more suppressions can outrank a small file with much higher density. Retain each file’s LOC and sort by
suppressions / LOC, as required by the feature objective.Also applies to: 2200-2207
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/agentready/assessors/code_quality.py` around lines 2159 - 2173, Update the suppression file-statistics flow around _count_file_suppressions to retain each file’s line count alongside its suppression count and rank files by suppression concentration (sup_count divided by line_count), rather than absolute suppression count. Apply the same change to the related statistics handling around the additional referenced section, while preserving existing test-file exclusion and zero-suppression behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/attributes.md`:
- Line 29: Update the attribute-count statements in docs/attributes.md to report
30 instead of 28, keeping them consistent with the 30-entry weights catalog and
the count stated on Line 6.
In `@src/agentready/models/config.py`:
- Around line 152-165: Unify lint-suppression configuration so the documented
assessor_options entry is not ignored: in src/agentready/models/config.py lines
152-165, expose one canonical path or validate/map the nested configuration into
LintSuppressionOptions; in src/agentready/assessors/code_quality.py lines
2061-2066, update the lint-suppression assessor to consume that canonical path
consistently; and in docs/attributes.md lines 1524-1532, document the same path
and configuration contract.
---
Duplicate comments:
In `@src/agentready/assessors/code_quality.py`:
- Around line 2080-2087: Update the test-file detection logic in the
JavaScript/TypeScript branch to derive test suffixes from
_LANG_EXTENSIONS[lang], covering JSX and TSX alongside JS and TS. Preserve the
existing directory and root-prefix checks, and add regression coverage
confirming Component.test.jsx and Component.spec.tsx are excluded when
exclude_tests=True.
- Around line 2159-2173: Update the suppression file-statistics flow around
_count_file_suppressions to retain each file’s line count alongside its
suppression count and rank files by suppression concentration (sup_count divided
by line_count), rather than absolute suppression count. Apply the same change to
the related statistics handling around the additional referenced section, while
preserving existing test-file exclusion and zero-suppression behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 0dbc6cd0-9e24-414d-aaaa-db0f3f0c65ae
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (9)
.agentready-config.example.yamldocs/attributes.mdsrc/agentready/assessors/__init__.pysrc/agentready/assessors/code_quality.pysrc/agentready/assessors/structure.pysrc/agentready/data/.agentready-config.example.yamlsrc/agentready/data/default-weights.yamlsrc/agentready/models/config.pytests/unit/test_assessors_code_quality.py
msu8
force-pushed
the
510
branch
2 times, most recently
from
July 17, 2026 06:59
cf2229e to
47fe732
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/agentready/assessors/code_quality.py`:
- Around line 2068-2094: Update _is_test_file to apply the shared
_TEST_DIR_FRAGMENTS and _TEST_ROOT_PREFIXES checks for every supported language
before language-specific filename rules. Add Java handling and ensure Java files
under test directories are excluded, while preserving existing Go, Python,
JavaScript, TypeScript, and Ruby filename detection; extend tests to cover the
additional supported languages.
- Around line 2111-2137: Update the file-discovery logic around
safe_subprocess_run and the os.walk fallback so Git failures, missing Git, or
timeouts do not trigger ignore-unaware traversal. Instead, return the assessor’s
established skipped/error finding for unavailable discovery, or replace the
fallback with an ignore-aware mechanism while preserving normal git ls-files
behavior.
- Around line 1937-1953: Extend _SUPPRESSION_PATTERNS for Python to recognize
whole-file # ruff: noqa and # flake8: noqa directives, and for TypeScript to
recognize // `@ts-nocheck` and // `@ts-expect-error`. Add regression tests covering
each directive and confirming they contribute to suppression density.
- Around line 2121-2125: The source-file discovery loop should reject symlinks
and other non-regular files before yielding paths, and downstream assessment
reads must be bounded. Update the logic around the visible rel_path filtering
and its corresponding read path to validate regular files without following
symlinks, then stream each file with a strict maximum byte limit instead of
unbounded read_text().
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: fbd34c4b-2f0e-4211-8ac8-260ab1093670
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (9)
.agentready-config.example.yamldocs/attributes.mdsrc/agentready/assessors/__init__.pysrc/agentready/assessors/code_quality.pysrc/agentready/assessors/structure.pysrc/agentready/data/.agentready-config.example.yamlsrc/agentready/data/default-weights.yamlsrc/agentready/models/config.pytests/unit/test_assessors_code_quality.py
Heavy use of suppression directives (//nolint, # noqa, eslint-disable, @SuppressWarnings, etc.) degrades lint as a quality signal for AI agents — lint passes, but not because the code is clean. New LintSuppressionAssessor (Tier 3, 2% weight) scans source files for suppression directives across 10 languages, normalized per 1,000 LOC: - ≤5/1k → score 100, pass - 5–15/1k → linear 100→0, fail - ≥15/1k → score 0, fail Thresholds and test-file exclusion are configurable via .agentready-config.yaml::assessor_options.lint_suppression_density. Closes ambient-code#510
Contributor
Author
|
Hi @jwm4, could you PTAL? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add
LintSuppressionAssessorto detect overuse of lint suppression directivesacross the codebase. A repo can have a comprehensive lint config and still
render it meaningless through blanket
//nolint,# noqa, or// eslint-disableusage — lint passes, but not because the code is clean.This gives AI agents a false signal that the codebase is healthy.
Type of Change
Related Issues
Fixes #510
Changes Made
LintSuppressionAssessor(lint_suppression_density, Tier 3, 2% weight)//nolint), Python (# noqa,# type: ignore,# pylint: disable), JavaScript/TypeScript (// eslint-disable,// @ts-ignore), Ruby (# rubocop:disable), Java (@SuppressWarnings), Terraform (# tflint-ignore:), Shell (# shellcheck disable=), Dockerfile (# hadolint ignore=), YAML (# yamllint disable,# kube-linter disable), Markdown (<!-- markdownlint-disable -->).agentready-config.yaml::assessor_optionsvendor/,node_modules/, and other generated directories from scanningdocs/attributes.mdupdated with scoring rules, language coverage, and remediation examplesdefault-weights.yamlupdated (v2.5.0, 30 total attributes)code_quality.py— no separate moduleTesting
pytest)Manual test: assessed
psf/black(Python),golangci/golangci-lint(Go) and othersChecklist
Additional Notes
Summary by CodeRabbit