Skip to content

Commit a7b0d00

Browse files
v5: fix partial-stage repair bug, cut redundant downloads (#292)
* Fix apply of created files from diff caches A diff archive has no delta for a file the patch creates, yet the disk stager counted a cached diff archive as covering the whole patch. With only diffs on disk, `apply --offline` passed the gate, patched the modified files, then failed on the created file's missing blob and left the package half-patched; online `apply` never fetched that blob at all. Coverage is now per file: a diff covers only files with a before-hash, and created files need their blob. Online, a cached diff archive no longer suppresses the download, and the top-up fetches just the created files' blobs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB * Batch vendor package-reference requests A vendor run asked the patch service for each package's download reference in its own request, though the endpoint takes 500 uuids at once: N round trips and N quota units for N packages. The run's download plan now resolves every planned uuid in one request, sent by the first planned call in place of its own and with the same retries, so an outage costs what it did before. Each package takes its answer from that batch at its turn; one still building is asked again then, as before. Hosted scan's reference lookup is chunked at the endpoint's 500-uuid cap, which it used to exceed with a 400. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB * Skip downloading pypi sdists vendoring rejects pypi vendoring is wheel-based, yet a pypi patch the service serves as an sdist (every patch without a file qualifier) was downloaded in full, then rejected because it is not a .whl. The service's reference already names the artifact, so a pypi reference whose artifact is not a wheel is now refused before the download, in the vendor loop and in its download plan alike. The outcome is unchanged: `auto` warns and builds the wheel locally, and `service` refuses. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB * Repair downloads created files' blobs A default (diff-mode) `repair` downloaded only diff archives, but a diff has no delta for a file the patch creates. After such a repair `apply --offline` still could not apply a patch that creates files. In diff mode, repair now also downloads the blobs of created files (and lists them under `--offline` and `--dry-run`), reported as their own blob download. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB * Defer pristine fetches the service makes moot With the patch service on, `vendor` deferred the registry download of a not-installed package only for cargo; npm, golang and composer packages were downloaded and verified up front even when the service's prebuilt artifact made the pristine copy unnecessary. Those backends also ask the service first and read the pristine tree only on a local-build fallback, so their download is now deferred the same way. A package is deferred only when its fetch would really download: the fetchers' pre-download refusals (a foreign yarn berry cacheKey, a go module go fetches without a proxy, a composer entry with no dist URL) are now one shared check that both the fetch and the deferral use. pypi and gem keep the up-front fetch, which their installed-variant probe reads. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB * Run the vlt get e2e leg in agent mode A bare `get` defaults to hosted mode since v5, so the real-vlt get_and_remove leg found the installed copy unpatched. Pass `--mode agent` as #283 does, which this ports. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB * Address review of the waste-review fixes - repair --json no longer counts created-file blobs twice (once under the diff-mode event); the closing line names both failure counts when both passes fail. - The vendor reference batch names the plan from the first call's position on, so a package the loop passed over is never granted. - The npm and yarn classic registry views no longer take a non-http resolution's integrity (a local tarball's hash, a git commit id) as a registry integrity, so such a package is never deferred behind, or vendored from, the service's registry build. - CHANGELOG entries for the new behavior, and the repair event row in CLI_CONTRACT. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB * Keep yarn git deps out of the registry view A yarn classic block resolved to a git repository over plain https (`https://…/repo.git#<commit>`, or a codeload tarball) passed as a registry tarball: its commit id became a sha1 integrity, and an `integrity` field on any git block was kept. With npm now deferring behind the patch service, such a lockfile-only git dependency could be vendored from the service's registry build instead of refusing `vendor_fetch_unverifiable`. A git resolution, over any protocol, now carries no URL and no integrity in the registry view, like npm's non-registry entries. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzdTEhubve9yWfqBE7vAsB --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent f9cb7e1 commit a7b0d00

16 files changed

Lines changed: 1450 additions & 171 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 41 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1183,8 +1183,10 @@ into the new version's section — see docs/releasing.md.
11831183
a server-side archive build and count against quota, are exactly the
11841184
one-at-a-time loop's (71 on a fresh depscan run, where an earlier
11851185
draft of the look-ahead issued 74). What changes is only their timing:
1186-
up to four are in flight at once. `SOCKET_API_CONCURRENCY=1` turns the
1187-
look-ahead off entirely.
1186+
they are requested in one batch at the first planned package (see
1187+
"Fewer downloads in vendored runs"), and up to four archives are in
1188+
flight at once. `SOCKET_API_CONCURRENCY=1` turns the look-ahead off
1189+
entirely.
11881190
- A token revoked *mid-run* now costs the authenticated batch endpoint
11891191
the requests already in flight — up to the in-flight cap instead of
11901192
one — before the run downgrades to the public proxy. Their answers are
@@ -1194,6 +1196,21 @@ into the new version's section — see docs/releasing.md.
11941196

11951197
### Fixed
11961198

1199+
- **`apply` no longer half-applies a patch that creates a file from a
1200+
diff-only cache.** A diff archive has no delta for a file the patch
1201+
creates, but the source check counted a cached diff archive as covering
1202+
the whole patch: `apply --offline` passed it, patched the modified files,
1203+
then failed on the created file's missing blob; online `apply` never
1204+
fetched that blob. Coverage is now per file (a diff covers only files
1205+
with a `beforeHash`), so `apply --offline` reports the patch as having no
1206+
local source up front and changes nothing, online `apply` fetches just
1207+
the created files' blobs, and a default (diff-mode) `repair` downloads
1208+
them too. Such a repair's `--json` envelope carries a second
1209+
`downloaded` (dry-run `verified`) artifact event with `mode: "file"` for
1210+
those blobs.
1211+
- **Hosted `scan` resolves more than 500 patches.** The package-reference
1212+
request is sent in chunks of 500 uuids, the endpoint's limit; a larger
1213+
scan used to fail with a 400.
11971214
- **`rollback` fetches a before-blob that only a store peer variant
11981215
needs.** The before-blob gate now probes every pnpm and vlt store variant
11991216
copy the rollback restores, so an online rollback no longer fails
@@ -1817,6 +1834,15 @@ into the new version's section — see docs/releasing.md.
18171834

18181835
### Changed
18191836

1837+
- **Fewer downloads in vendored runs.** A vendored run now asks the patch
1838+
service for all of its planned packages' download references in one
1839+
request (in chunks of 500) from the first package it reaches, in place
1840+
of one request per package; an outage costs the same retries as before,
1841+
and a package the service reports still building is asked again at its
1842+
turn. A pypi patch the service serves as an sdist (every patch without a
1843+
file qualifier) is refused from its reference, before its bytes are
1844+
downloaded: `auto` still warns `vendor_prebuilt_unavailable` and builds
1845+
the wheel locally, `service` still refuses.
18201846
- **One owner rule for the patch stores.** `list`, `vex`, `scan`'s
18211847
`updates[]`, `rollback` and `remove` now read the
18221848
manifest and the vendor ledger (plus, in `vex`, the hosted records)
@@ -1927,22 +1953,25 @@ into the new version's section — see docs/releasing.md.
19271953
covers the purl (its entry records the record's patch uuid and the
19281954
committed artifact is on disk — a file artifact such as a wheel or
19291955
tarball only while it still hashes to the ledger's `sha256`; `--force`
1930-
keeps the eager fetch), and for every lockfile-only cargo crate the
1931-
registry could fetch and verify (a crates.io `Cargo.lock` entry with a
1932-
checksum, or the pre-vendor resolution the ledger recovers) while the
1933-
patch service is enabled (the cargo backend reads the pristine source
1934-
only once `cargo_service_copy` falls back to the local build). A git,
1935-
path or custom-registry crate is never deferred: it keeps the eager
1936-
ladder's `vendor_fetch_unverifiable` + `package_not_installed` refusal
1937-
and is not vendored from the service's crates.io build, and a committed
1956+
keeps the eager fetch), and for every lockfile-only npm, cargo, golang
1957+
or composer package the registry would fetch and verify (a lock entry
1958+
with an integrity, or the pre-vendor resolution the ledger recovers,
1959+
that none of its fetcher's pre-download refusals applies to) while the
1960+
patch service is enabled (those backends read the pristine source only
1961+
once the service falls back to the local build; pypi and gem keep the
1962+
eager fetch, which their installed-variant probe reads). A git, path,
1963+
local-tarball or custom-registry package is never deferred: it keeps the
1964+
eager ladder's `vendor_fetch_unverifiable` + `package_not_installed`
1965+
refusal and is not vendored from the service's registry build, and a committed
19381966
file artifact that no longer matches its pin keeps the eager ladder's
19391967
outcome too. Visible effects: an idempotent re-run
19401968
makes no registry requests and no longer reports `vendor_fetched_missing`
19411969
for fetches it never needed; with no network (or under `--offline`) the
19421970
re-run of an already-vendored pypi, cargo, go or lockfile-only gem
19431971
project now SUCCEEDS (`already_vendored`, exit 0) instead of failing
1944-
`vendor_fetch_failed` / `package_not_installed`; a cargo crate the service
1945-
serves is never downloaded from the registry. When a deferred fetch does
1972+
`vendor_fetch_failed` / `package_not_installed`; an npm, cargo, golang or
1973+
composer package the service serves is never downloaded from the
1974+
registry. When a deferred fetch does
19461975
happen (a drifted committed copy being rebuilt locally, a service miss),
19471976
its `vendor_fetched_missing` warning is recorded just ahead of that
19481977
package's own event instead of in the up-front fetch pass, and a failed,

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)