feat: drop shadow-diff body mismatches on records edited after capture (CM-1473)

[mbani01]

Summary

Follow-up to #4895. ~85% of the remaining shadow-diff field_mismatch entries are comment / PR / issue bodies that were edited after one side captured them (GitHub lastEditedAt confirmed on every sampled record from the 2026-10-06 characterisation). Both sides are correct for their capture time, so these are revision drift, not data gaps.

This adds a confirmation step mirroring the force-push and deleted-record checks:

• Candidates: field_mismatch entries whose only differing field is body, on any sync except pull-request-commits (commit messages are immutable).
• Each candidate resolves to a GitHub node id: its own sourceId, or the parent PR / issue embedded in synthetic gen-… timeline ids (review-requested, closed, merged, assigned, issues-closed). Parents are queried once.
• Batched nodes(ids) { ... on Comment { lastEditedAt } } (100 ids per request, 3 concurrent, 60s budget). A non-null lastEditedAt drops the mismatch; the shadow row is then pruned as matched by the existing persistence.
• Failed batches and null nodes keep their mismatches visible and are logged as check_failed with the count, unlike the deleted-record check which excludes unchecked candidates.

Diff-only, no sync behaviour change. Expected cost: a few GraphQL requests per nightly run across the fleet.

Test plan

• pnpm vitest run services/apps/connectors_worker/src (72 passed, 10 new in editedRecords.test.ts)
• oxlint --deny-warnings, oxfmt --check, tsc --noEmit on connectors_worker
• Replayed real 2026-10-06 mismatches through candidate detection: synthetic PR/issue ids resolve correctly, commits excluded, never-edited vscode issue bodies stay visible
• After deploy: fleet fieldMismatchCount in sync_diff_summary drops from ~560/day to the commit-stats + Nango-rendering remainder; confirmedEditedCount visible in worker logs

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[cursor]

PR Summary

Medium Risk
Changes nightly shadow-diff classification and adds GitHub GraphQL calls with time budgets; failures keep mismatches visible but confirmed drops affect reported fieldMismatchCount and shadow pruning.

Overview
Adds a post-diff GitHub confirmation for shadow-diff noise from body-only field_mismatch rows where GitHub later edited the comment/PR/issue after capture.

New editedRecords logic treats candidates as mismatches that differ only on body (excluding pull-request-commits), resolves real or synthetic gen-… timeline ids to a parent node id, and batches GraphQL nodes { lastEditedAt } checks. Confirmed edits are dropped from the mismatch set (so existing pruning can treat them as matched); failed/null lookups stay as field_mismatch, unlike unchecked deleted-record candidates.

runShadowDiffForChannel runs this alongside force-push and deleted-record checks, reuses the shared GitHub HTTP client, and updates setup-failure logging so edited candidates remain visible when confirmation cannot run.

^{Reviewed by Cursor Bugbot for commit d2abd8d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

🟡 Changes recommended

Edit confirmation is overbroad and can suppress genuine mismatches while missing some failure telemetry.

2 open findings

• Restrict synthetic resolution to parent-snapshot event types · New
• Count edited candidates when GitHub client setup fails · New

What changed in this PR

Adds GitHub edit confirmation to reduce false-positive shadow-diff body mismatches.

Changes:

• Detects body-only mismatch candidates and queries GitHub in batches.
• Removes confirmed edited records and logs confirmation outcomes.
• Adds unit coverage for candidate filtering and request failures.

File	Description
editedRecords.ts	Implements edited-record detection and filtering.
editedRecords.test.ts	Tests candidate and confirmation behavior.
shadowDiffActivities.ts	Integrates filtering and operational logging.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

Null-node failures are not reported, and historical edits can suppress genuine body mismatches.

0 open findings

2 resolved since last review

• Restrict synthetic resolution to parent-snapshot event types
• Count edited candidates when GitHub client setup fails

Previously missed (2)

In code that hasn't changed since last review

Use snapshot timestamps when filtering edited nodes

services/​apps/​connectors_worker/​src/​editedRecords.ts:86

A non-null lastEditedAt only proves that this node was edited at some point; it does not prove the edit happened after either side captured it. An issue/PR/comment edited before both snapshots will still enter editedNodeIds, so any genuine body-mapping mismatch on that record is silently dropped and its shadow row pruned. Carry the relevant capture timestamp into this check and suppress only when lastEditedAt is later than the snapshot that could contain the stale body.

Count null node results as unchecked

services/​apps/​connectors_worker/​src/​editedRecords.ts:86

Null node results are treated as a successful “not edited” check here: the mismatch is retained, but its ID is never added to uncheckedNodeIds, so unconfirmedCount stays zero and the promised check_failed warning is omitted. Return null/partial node IDs separately from edited IDs and include them in the unchecked count, as the PR description specifies.

🧠 Review effort: Balanced

[mbani01]

On the two "previously missed" notes from the Copilot re-review:

Count null node results as unchecked — fixed in 95e3b1c. Null nodes now go into the unchecked set, so they are kept as field_mismatch and counted in unconfirmedCount / the check_failed warning, matching the PR description. Test updated.

Use snapshot timestamps when filtering edited nodes — not doable in this PR. Neither side stores a capture timestamp: shadow rows only carry the activity's occurredAt (creation time) and Nango metadata holds sync times, not GitHub edit times. Adding a capture timestamp means changing what the syncs emit, which is out of scope for a diff-only change. Residual risk is a record that was edited before both captures and has a genuine body-mapping bug; a mapping bug would also surface on the never-edited majority of records, so it is not hidden by this rule.

[Copilot]

🟡 Changes recommended

lastEditedAt does not establish that an edit occurred between the two captures, allowing genuine mismatches to be suppressed.

1 open finding

• Filter edits by capture window to avoid hiding persistent mismatches · New

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Partial GraphQL errors can bypass the required check_failed accounting and warning.

1 open finding

• GraphQL partial errors omit check_failed warnings · New

1 resolved since last review

• Filter edits by capture window to avoid hiding persistent mismatches

🧠 Review effort: Balanced