Add Agent Host E2E coverage and expand protocol scenarios

[roblourens]

Summary

• collect opt-in native V8 coverage from Agent Host child processes and generate text, HTML, LCOV, JSON, and a checked-in loaded-files-only baseline
• add 50 focused protocol integration tests plus 15 shared real-provider E2E scenarios backed by recorded LLM fixtures and AHP snapshots
• harden record/replay normalization across temp workspaces, macOS realpaths, fixed legacy suffixes, and Windows JSON paths
• use compact behavior snapshots for filesystem/shell scenarios: real tools still execute, while raw output, display strings, repeated ready events, and incidental session updates are excluded
• fix resource protocol behavior found by the new tests, including missing-file append/insert, parent validation, conflict classification, serialized writes, dirty-write checks, and exclusive createOnly writes
• document the coverage workflow and the three test layers in this folder

Coverage remains informational; this PR does not add a threshold or commit gate.

Coverage

Final aggregate loaded-file baseline:

• lines/statements: 71.53% (60,292 / 84,285)
• branches: 64.79% (5,296 / 8,174)
• functions: 60.34% (1,925 / 3,190)

The report includes loaded executable files under src/vs/platform/agentHost/{common,node} and excludes tests, provider dependencies, generated type-only modules, and unloaded files.

Expected-behavior skips found while adding E2E scenarios

The tests remain present but only the affected provider variants are skipped:

• Copilot: existing-file read, edit, JSON read, and delete turns do not complete
• Codex: JSON and line-count responses duplicate the requested value
• Codex: create-file, nested-create, and shell snapshots have nondeterministic serverToolsChanged ordering
• Claude/Codex: git-status snapshots have nondeterministic customization/changeset ordering

Record-only, provider capability, and steering skips that predate these scenarios remain unchanged.

Validation

• npm run typecheck-client
• npm run test-agent-host-e2e-coverage
  • resource group: 41 passing
  • protocol group: 78 passing
  • provider replay group: 51 passing, 35 pending
• npm run valid-layers-check
• npm run precommit
• fixture/snapshot privacy scan for absolute user paths, usernames, and credentials

(Written by Copilot)

[Copilot]

Pull request overview

Adds Agent Host coverage tooling, broader protocol tests, and recorded cross-provider E2E scenarios while hardening resource writes and snapshot normalization.

Changes:

• Collects native V8 coverage and checks in loaded-file metrics.
• Adds protocol/resource tests and Claude, Codex, and Copilot replay scenarios.
• Improves resource-write concurrency and fixture/path normalization.

Show a summary per file

File	Description
.gitignore	Allows the checked-in coverage baseline.
package.json	Adds the coverage command and c8.
package-lock.json	Locks the c8 dependency.
scripts/agent-host-e2e-coverage.ts	Runs tests and generates coverage reports.
agentService.ts	Hardens resource-write behavior.
agentHostE2ETestHelpers.ts	Defines shared provider scenarios and lifecycle.
ahpSnapshot.ts	Adds code-driven snapshots and path normalization.
testHelpers.ts	Adds coverage environment and timeout support.
resourceOperations.integrationTest.ts	Expands resource protocol coverage.
README.md	Documents test layers and coverage workflow.
coverage/agentHostE2E.json	Records aggregate and per-file coverage.
agentHostServer.integrationTest.ts	Adapts startup timeout for coverage.
clientTools.integrationTest.ts	Adapts startup timeout for coverage.
handshake.integrationTest.ts	Adds handshake/error scenarios.
multiClient.integrationTest.ts	Adapts startup timeout for coverage.
networkDiagnostics.integrationTest.ts	Adapts startup timeout for coverage.
otlpLogs.integrationTest.ts	Adapts startup timeout for coverage.
sessionConfig.integrationTest.ts	Expands coverage-mode timeouts.
sessionDiffs.integrationTest.ts	Adapts startup timeout for coverage.
sessionFeatures.integrationTest.ts	Adapts startup timeout for coverage.
sessionLifecycle.integrationTest.ts	Adapts startup timeout for coverage.
toolApproval.integrationTest.ts	Adapts startup timeout for coverage.
turnExecution.integrationTest.ts	Tests missing chats and invalid cursors.
captures/agentHostE2E/claude-*.yaml	Adds Claude model replay fixtures.
captures/agentHostE2E/codex-*.yaml	Adds Codex model replay fixtures.
captures/agentHostE2E/copilotcli-*.yaml	Adds Copilot model replay fixtures.
__snapshots__/Agent_Host_E2E___Claude_*.yaml	Adds Claude AHP expectations.
__snapshots__/Agent_Host_E2E___Codex_*.yaml	Adds Codex AHP expectations.
__snapshots__/Agent_Host_E2E___Copilot_*.yaml	Adds Copilot AHP expectations.

Review details

Comments suppressed due to low confidence (1)

src/vs/platform/agentHost/test/node/protocol/agentHostE2ETestHelpers.ts:1068

• This explicitly shell-dependent test still runs on Windows, although its fixtures invoke Bash/bash and this file documents POSIX shell scenarios as unsupported there. Gate it with !isWindows, as the worktree test already does, otherwise Windows replay can select a different shell/tool and time out or miss the fixture.

• Files reviewed: 95/98 changed files
• Comments generated: 3
• Review effort level: Medium