Skip to content

Commit e31da36

Browse files
mikolalysenkoclaude
andcommitted
fix(core): hosted/vendored rewriter fixes found by the real-PM matrices
First of seven commits that land the manifest-less VEX branch (feat/vex-lockfile-inventory) on top of #247. The branch's ~165-commit history, built on the pre-#247 main 9489b18 with per-PM merge commits, was squashed onto 09956d9 and re-split by concern; the full pre-rebase history is preserved on branch backup/vex-lockfile-pre-rebase. These are the product bugs the per-PM real-toolchain matrices turned up while the manifest-less VEX suites were written. Each is independent of VEX and pinned by a core regression test: - composer (hosted): the rewriter drops the entry's `source` block when it immediately precedes `dist` (one `redirect_composer_dist` fragment edit spanning both, reverted byte-for-byte). Composer 1 and 2.2 LTS silently installed the pristine upstream commit from git whenever the hosted download failed; a hand-ordered source that cannot be dropped warns `redirect_composer_source_kept`. Golden fixture: `source-and-dist`. - gem (hosted): the patch-registry `GEM` section is inserted in bundler's source order (sorted by remote), so a frozen `bundle install` on bundler >= 4.0.19 accepts the converged lock. The `basic` golden and the exact lock expectations move the Socket section first. - cargo (hosted + vendored): a v1 `Cargo.lock` (checksums in `[metadata]`, dependents naming the crate by its full package id) is redirected and vendored correctly: the `[metadata]` line and every dependent's full-id reference follow the source, each fragment its own ledger edit, and `cargo --locked` accepts the result; the vendored detach/restore of a v1 entry is byte-identical. `plan_cargo_lock` keeps #247's multi-source twin disambiguation (`Ambiguous`) and hoisted regexes; the block end now also stops at a trailing `[metadata]` / `[[patch.unused]]`, and the checksum is inserted after a block-final `source` line too. - yarn berry (hosted + vendored): written checksums follow the lock's own spelling — yarn 4.0.0–4.0.2 spell `10c0` checksums as bare hex, so the prefixed form failed `yarn install --immutable` with YN0028. - npm (vendored): npm 12 reifies from the `package-lock.json` it creates beside a committed shrinkwrap, so `vendor` now rewires every present npm lock (siblings first, primary last; an unrewirable sibling warns `vendor_npm_sibling_lock_unwired`), and the in-use/revert probes read every npm lock before deleting an artifact. `select_lockfile` reads through #247's guarded `read_regular_to_bytes`. - npm (hosted): a lockfileVersion 1 redirect warns `redirect_npm_legacy_client` — npm 6 ignores a v1 lock's `resolved` and fails EINTEGRITY against the patched pin. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent e024629 commit e31da36

11 files changed

Lines changed: 1416 additions & 168 deletions

File tree

‎crates/socket-patch-core/src/patch/redirect/mod.rs‎

Lines changed: 668 additions & 61 deletions
Large diffs are not rendered by default.

‎crates/socket-patch-core/src/vendor/cargo_lock.rs‎

Lines changed: 136 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,17 @@
2323
//! `version = 4` line keep their exact bytes — zero formatting churn in the
2424
//! committed diff.
2525
//!
26+
//! Lock format v1 (cargo < 1.41, still read by every cargo and never
27+
//! rewritten under `--locked`) spells the pair differently: the entry holds
28+
//! only `source`, the checksum sits in the trailing `[metadata]` table as
29+
//! `"checksum <name> <version> (<source>)"`, and dependents reference the
30+
//! crate by that full `"<name> <version> (<source>)"` id. Detaching there
31+
//! also drops the `[metadata]` key and rewrites the references to the
32+
//! sourceless `"<name> <version>"` form cargo v1 uses for path packages —
33+
//! otherwise they name a package the lock no longer has and cargo refuses
34+
//! the lock under `--locked` (real cargo 1.93: "cannot update the lock
35+
//! file … because --locked was passed"). Restore reverses all three.
36+
//!
2637
//! The removed `source`/`checksum` pair is not recoverable offline (the
2738
//! checksum is the sha256 of the registry `.crate` tarball, not of the
2839
//! extracted tree), so [`detach_lock_entry`] returns it as the vendor ledger's
@@ -99,6 +110,39 @@ fn find_package_mut<'a>(
99110
})
100111
}
101112

113+
/// The `[metadata]` key a v1 lock files `name`+`version`'s checksum under.
114+
fn metadata_checksum_key(name: &str, version: &str, source: &str) -> String {
115+
format!("checksum {name} {version} ({source})")
116+
}
117+
118+
/// A v1 lock: no top-level `version` key and a `[metadata]` table (kept,
119+
/// even emptied, by [`detach_lock_entry`] — so a detached v1 lock still
120+
/// reads as v1 on restore).
121+
fn is_v1_lock(doc: &DocumentMut) -> bool {
122+
doc.get("version").is_none() && doc.get("metadata").is_some_and(Item::is_table_like)
123+
}
124+
125+
/// Rewrite every `dependencies` entry spelled exactly `from` to `to`,
126+
/// keeping each entry's formatting.
127+
fn rewrite_dependency_refs(doc: &mut DocumentMut, from: &str, to: &str) {
128+
let Some(pkgs) = doc
129+
.get_mut("package")
130+
.and_then(Item::as_array_of_tables_mut)
131+
else {
132+
return;
133+
};
134+
for pkg in pkgs.iter_mut() {
135+
let Some(deps) = pkg.get_mut("dependencies").and_then(Item::as_array_mut) else {
136+
continue;
137+
};
138+
for i in 0..deps.len() {
139+
if deps.get(i).and_then(toml_edit::Value::as_str) == Some(from) {
140+
deps.replace(i, to);
141+
}
142+
}
143+
}
144+
}
145+
102146
/// Commit the edited lock atomically (stage + fsync + rename). The lock is a
103147
/// committed file shared with cargo itself; a torn write would corrupt the
104148
/// whole project's resolution, so never truncate-in-place. Mode-preserving:
@@ -132,14 +176,28 @@ pub async fn detach_lock_entry(
132176
Some(s) => s.to_string(),
133177
None => return Err(LockEditError::NotRegistry),
134178
};
135-
let checksum = table
179+
let mut checksum = table
136180
.get("checksum")
137181
.and_then(Item::as_str)
138182
.map(str::to_string);
139183

140184
table.remove("source");
141185
table.remove("checksum");
142186

187+
// v1: the checksum lives in `[metadata]`, and dependents name the crate
188+
// by its full id (any format may spell an ambiguous ref that way).
189+
let key = metadata_checksum_key(name, version, &source);
190+
if let Some(meta) = doc.get_mut("metadata").and_then(Item::as_table_like_mut) {
191+
if let Some(sum) = meta.remove(&key) {
192+
checksum = checksum.or_else(|| sum.as_str().map(str::to_string));
193+
}
194+
}
195+
rewrite_dependency_refs(
196+
&mut doc,
197+
&format!("{name} {version} ({source})"),
198+
&format!("{name} {version}"),
199+
);
200+
143201
if !dry_run {
144202
write_lock(&path, &doc).await?;
145203
}
@@ -159,6 +217,7 @@ pub async fn restore_lock_entry(
159217
dry_run: bool,
160218
) -> Result<bool, LockEditError> {
161219
let (path, mut doc) = read_lock(project_root).await?;
220+
let v1 = is_v1_lock(&doc);
162221
let Some(table) = find_package_mut(&mut doc, name, version) else {
163222
return Ok(false);
164223
};
@@ -167,7 +226,7 @@ pub async fn restore_lock_entry(
167226
}
168227

169228
table.insert("source", toml_edit::value(original.source.as_str()));
170-
if let Some(checksum) = &original.checksum {
229+
if let (Some(checksum), false) = (&original.checksum, v1) {
171230
table.insert("checksum", toml_edit::value(checksum.as_str()));
172231
}
173232
// `insert` appends, but cargo's canonical key order is
@@ -183,6 +242,26 @@ pub async fn restore_lock_entry(
183242
};
184243
table.sort_values_by(|k1, _, k2, _| rank(k1.get()).cmp(&rank(k2.get())));
185244

245+
if v1 {
246+
// Back into `[metadata]` (cargo writes its keys sorted) and the
247+
// dependents' references back to the full id.
248+
if let (Some(checksum), Some(meta)) = (
249+
&original.checksum,
250+
doc.get_mut("metadata").and_then(Item::as_table_mut),
251+
) {
252+
meta.insert(
253+
&metadata_checksum_key(name, version, &original.source),
254+
toml_edit::value(checksum.as_str()),
255+
);
256+
meta.sort_values();
257+
}
258+
rewrite_dependency_refs(
259+
&mut doc,
260+
&format!("{name} {version}"),
261+
&format!("{name} {version} ({})", original.source),
262+
);
263+
}
264+
186265
if !dry_run {
187266
write_lock(&path, &doc).await?;
188267
}
@@ -300,6 +379,61 @@ mod tests {
300379
dir
301380
}
302381

382+
/// Cargo.lock v1: checksum in `[metadata]`, dependents referencing the
383+
/// crate by full id. REGRESSION: detach removed only the entry's
384+
/// `source`, leaving `"cfg-if 1.0.4 (registry+…)"` references (and the
385+
/// `[metadata]` checksum) naming a package the lock no longer has —
386+
/// real cargo then refuses the vendored lock under `--locked`.
387+
#[tokio::test]
388+
async fn detach_and_restore_a_v1_lock_follow_metadata_and_full_id_refs() {
389+
let other = "a".repeat(64);
390+
let v1 = format!(
391+
"[[package]]\nname = \"app\"\nversion = \"0.1.0\"\ndependencies = [\n \
392+
\"cfg-if 1.0.4 ({SOURCE})\",\n \"log 0.4.20 ({SOURCE})\",\n]\n\n\
393+
[[package]]\nname = \"cfg-if\"\nversion = \"1.0.4\"\nsource = \"{SOURCE}\"\n\n\
394+
[[package]]\nname = \"log\"\nversion = \"0.4.20\"\nsource = \"{SOURCE}\"\n\
395+
dependencies = [\n \"cfg-if 1.0.4 ({SOURCE})\",\n]\n\n\
396+
[metadata]\n\"checksum cfg-if 1.0.4 ({SOURCE})\" = \"{CHECKSUM}\"\n\
397+
\"checksum log 0.4.20 ({SOURCE})\" = \"{other}\"\n"
398+
);
399+
let dir = tempfile::tempdir().unwrap();
400+
let lock = dir.path().join("Cargo.lock");
401+
tokio::fs::write(&lock, &v1).await.unwrap();
402+
403+
let orig = detach_lock_entry(dir.path(), "cfg-if", "1.0.4", false)
404+
.await
405+
.unwrap();
406+
assert_eq!(orig.source, SOURCE);
407+
assert_eq!(
408+
orig.checksum.as_deref(),
409+
Some(CHECKSUM),
410+
"read from [metadata]"
411+
);
412+
let detached = tokio::fs::read_to_string(&lock).await.unwrap();
413+
assert_eq!(
414+
detached,
415+
format!(
416+
"[[package]]\nname = \"app\"\nversion = \"0.1.0\"\ndependencies = [\n \
417+
\"cfg-if 1.0.4\",\n \"log 0.4.20 ({SOURCE})\",\n]\n\n\
418+
[[package]]\nname = \"cfg-if\"\nversion = \"1.0.4\"\n\n\
419+
[[package]]\nname = \"log\"\nversion = \"0.4.20\"\nsource = \"{SOURCE}\"\n\
420+
dependencies = [\n \"cfg-if 1.0.4\",\n]\n\n\
421+
[metadata]\n\"checksum log 0.4.20 ({SOURCE})\" = \"{other}\"\n"
422+
)
423+
);
424+
assert_eq!(
425+
probe_lock_entry(dir.path(), "cfg-if", "1.0.4").await,
426+
LockEntryProbe::Detached
427+
);
428+
429+
assert!(
430+
restore_lock_entry(dir.path(), "cfg-if", "1.0.4", &orig, false)
431+
.await
432+
.unwrap()
433+
);
434+
assert_eq!(tokio::fs::read_to_string(&lock).await.unwrap(), v1);
435+
}
436+
303437
#[tokio::test]
304438
async fn detach_removes_only_source_and_checksum() {
305439
let dir = fixture().await;

‎crates/socket-patch-core/src/vendor/npm_flavor.rs‎

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -393,8 +393,9 @@ pub async fn vendored_entry_in_use(entry: &VendorEntry, project_root: &Path) ->
393393
// The remaining flavors wire resolutions into the lock itself
394394
// (resolved URLs / file: ranges / package tuples), so a textual
395395
// probe for the uuid dir is exact: the path appears iff some
396-
// resolution still points at the artifact. shrinkwrap wins over
397-
// package-lock, mirroring the vendor/revert lockfile selection.
396+
// resolution still points at the artifact. Both npm locks are
397+
// probed: npm <= 11 installs from the shrinkwrap, npm 12 from the
398+
// package-lock beside it.
398399
None | Some("package-lock") => {
399400
lock_text_mentions_uuid(
400401
project_root,
@@ -431,21 +432,32 @@ pub async fn vendored_entry_in_use(entry: &VendorEntry, project_root: &Path) ->
431432
}
432433
}
433434

434-
/// First readable lockfile from `names`, probed for the uuid artifact dir.
435-
/// Shared with the textual backends' unwired-revert guard
435+
/// Every readable lockfile from `names`, probed for the uuid artifact dir:
436+
/// `Some(true)` when ANY of them mentions it, `Some(false)` when at least one
437+
/// was readable and none does, `None` when none was readable. Shared with
438+
/// the textual backends' unwired-revert guard
436439
/// ([`super::npm_lock::guard_unwired_textual_revert`]).
440+
///
441+
/// It used to stop at the FIRST readable name (npm <= 11's shrinkwrap-wins
442+
/// rule), but npm 12 installs from package-lock.json beside a committed
443+
/// npm-shrinkwrap.json, so a mention in either lock can be the one an
444+
/// install resolves through.
437445
pub(super) async fn lock_text_mentions_uuid(
438446
project_root: &Path,
439447
names: &[&str],
440448
uuid: &str,
441449
) -> Option<bool> {
442450
let needle = format!(".socket/vendor/npm/{uuid}/");
451+
let mut any_readable = false;
443452
for name in names {
444453
if let Ok(text) = read_regular_to_string(&project_root.join(name)).await {
445-
return Some(text.contains(&needle));
454+
if text.contains(&needle) {
455+
return Some(true);
456+
}
457+
any_readable = true;
446458
}
447459
}
448-
None
460+
any_readable.then_some(false)
449461
}
450462

451463
/// Revert one recorded npm vendor entry through the flavor that wired it.

0 commit comments

Comments
 (0)