Skip to content

Fix dependency evidence recording for unchanged unlocked Python resolutions #2552

Closed

Description

@groupthinking

Observed failure

On November 2, dependency review on #2401, #2400, #2446 and #2398 reached the evidence recorder but rejected rows for an unchanged prototype requirements file. The complete captured comparison for base ffd07a and #2398 head d6fef80 contains 68 rows, including filelock 4.2.6 removed and 4.2.7 added under docs/knowledge_prototypes/universal-automation-service/requirements.txt. That source is the identical regular Git blob a671735c0cfe4cf9b7ae2c9f8b412acedcffcba1 at both commits and contains bare unpinned declarations. The unmodified recorder raises its unchanged-manifest error and saves no artifact.

This is a recorder classification defect, distinct from the snapshot-count mismatch completed in #2490 and the snapshot submission scope described by #2506. It does not establish that any underlying vulnerability finding is false.

Intended behavior

Record the complete dependency comparison and unchanged original review outcome when a narrowly supported resolver difference is proved from identical source. Classify only paired pip version changes with unchanged scope from the same regular requirements blob containing bare unpinned names. Record the source blob IDs, content hash, declared names, complete classified rows and the limitation that unchanged source does not prove reproducible resolution.

Reject unsupported, malformed, missing, changed or ambiguous evidence. Preserve exact npm/uv dependency identity, version, scope and completeness checks, pagination trust/cycle checks, snapshot-warning rejection, and existing vulnerability/license policy. Do not drop raw rows or turn a failed policy review into success.

Acceptance criteria

  • Reproduce the original failure with the complete captured 68-row comparison and exact Git source identities.
  • Save schema-3 evidence with all raw rows byte-for-byte intact and the original review_outcome; retain genuine Vite scope changes and advisories.
  • Add negative coverage for attached hashes, malformed/unsupported versions, non-bare declarations, changed/missing/symlink sources, unpaired/duplicate/scope-changing rows, and incomplete lock evidence. Run the affected regression suites and Ruff against the exact candidate.
  • Independently review the final source, failed-before/passed-after cases and saved replay output.
  • Separately run a controlled manifest-changing PR off main with the exact recorder candidate. Verify exact base/head/run identity, successful snapshot submission, warning-free complete comparison, and the uploaded artifact's schema, raw count/hash/rows, source proof when present, and original policy verdict. A code-only green check does not meet this requirement. Fresh resolver drift may be absent; preserve the actual-input replay as separate evidence for that case.

Current evidence and boundaries

Local candidate c62c51f passes 101 affected tests, including 34 new cases, and Ruff. The independent reviewer reproduced the results and confirmed the full 68-row replay retains review_outcome=failure. Hosted artifact verification remains pending.

The repair PR implements this issue. A separate canary carries the exact fix and the already-tested #2400 manifest/lock bytes solely to exercise the real hosted workflow; it must remain unmerged. Its success will not establish that the original update PRs against main have passed. Keep all security findings enforced; no automatic approval, merge, deployment or policy waiver is requested.

Activity

  1. linear-code commented on Nov 2, 2026

    @linear-code
    Contributor
  2. github-actions commented on Nov 2, 2026

    @github-actions
    Contributor

    👋 Thanks for opening this issue! It has been automatically labeled for triage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions