From f36a803ab4c21a2015be927de834df3114746530 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 14:54:21 +0000 Subject: [PATCH 1/5] Find Hatch's out-of-tree project environments Hatch installs a project into envs under its data directory, never ./.venv, so stale-install checks, VEX and agent mode never looked at the environment hatch run actually uses. Model Hatch's placement rules (data dir, dirs.env.virtual, flat layouts, explicit env paths, the project id hash) and add those envs to local venv discovery. Hosted scans now warn about a stale Hatch env with the remedy that works (hatch env remove / prune), and vendored Hatch gets the same check as pypi_hatch_stale_install. A real-Hatch e2e covers both modes from an existing env through vex and the remedy. Fixes #335 Assisted-by: Claude Code:claude-opus-5-5 --- crates/socket-patch-cli/CLI_CONTRACT.md | 3 +- .../src/commands/scan/hosted/python.rs | 65 +- .../tests/e2e_vex_build/hatch.rs | 139 ++++ .../src/crawlers/hatch_env.rs | 672 ++++++++++++++++++ crates/socket-patch-core/src/crawlers/mod.rs | 1 + .../src/crawlers/python_crawler.rs | 63 +- crates/socket-patch-core/src/vendor/pypi.rs | 197 ++++- 7 files changed, 1112 insertions(+), 28 deletions(-) create mode 100644 crates/socket-patch-core/src/crawlers/hatch_env.rs diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 69fb09df9..6e8c61b79 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -189,7 +189,7 @@ The hidden alias `--no-apply` on `get --save-only` is **part of the contract** `repair` keeps its `gc` visible alias. -**Python stale-install guard**: after a hosted redirect, `scan` / `get` use the Python crawler to inspect every matching installed package, including Poetry's out-of-tree virtualenvs and `--global-prefix`. A readable file that differs from the patch's `afterHash` emits `redirect_pypi_stale_install` in JSON `redirect.warnings[]` and human stderr. The probe changes no installed files, re-runs on idempotent scans, and falls back to persisted patch records when fresh record fetching fails. Missing/unreadable files alone do not prove staleness; lock-only checkouts stay quiet. Dry runs skip the probe. Same-run VEX excludes positively stale Python packages (qualifier-insensitive), even with `--vex-no-verify` or a healthy copy in another interpreter; if nothing remains to attest, the command exits 1 with `no_applicable_patches`. Reinstall from the rewritten lock in the affected interpreter and verify with `socket-patch vex`. +**Python stale-install guard**: after a hosted redirect, `scan` / `get` use the Python crawler to inspect every matching installed package, including Poetry's out-of-tree virtualenvs, the project's Hatch environments (Hatch's data dir / `HATCH_DATA_DIR`, `[dirs.env] virtual`, explicit env `path`s) and `--global-prefix`. A readable file that differs from the patch's `afterHash` emits `redirect_pypi_stale_install` in JSON `redirect.warnings[]` and human stderr. The probe changes no installed files, re-runs on idempotent scans, and falls back to persisted patch records when fresh record fetching fails. Missing/unreadable files alone do not prove staleness; lock-only checkouts stay quiet. Dry runs skip the probe. Same-run VEX excludes positively stale Python packages (qualifier-insensitive), even with `--vex-no-verify` or a healthy copy in another interpreter; if nothing remains to attest, the command exits 1 with `no_applicable_patches`. Reinstall from the rewritten lock in the affected interpreter and verify with `socket-patch vex`. A stale Hatch environment instead names `hatch env remove ` / `hatch env prune`: Hatch's pip installer (and uv before Hatch 1.16) keeps a same-version release, so only a recreated env picks up the patch. The same Hatch envs are what agent mode patches and `vex` judges for a Hatch project. ### socket.yml patch policy (v5.0) @@ -1234,6 +1234,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `pypi_poetry_symlink_unsupported` / `pypi_pipenv_symlink_unsupported` / `pypi_requirements_symlink_unsupported` | `failed` | vendor (pypi, v5.0): a target file (`pyproject.toml` / `poetry.lock`, `Pipfile` / `Pipfile.lock`, or any planned `requirements*.txt`) is a symlink — refused before any write on wire AND on revert (the revert keeps the artifact, `kept_artifact`); the twins of the existing pdm/uv symlink refusals. | | `pypi_poetry_changed` / `pypi_pdm_changed` / `pypi_pipenv_changed` / `pypi_uv_changed` | `failed` | vendor (pypi, v5.0): the lock / project file changed between the read that planned the edit and the first write — refused before any write (worded like `pypi_lock_changed`: " changed during vendoring; re-run"). | | `pypi_pipenv_stale_install` | `skipped` (warning) | vendor (pipenv): the vendored twin of `redirect_pypi_stale_install` — the project's venv still holds the upstream release Pipenv will not reinstall over; the detail names the `pipenv run pip uninstall -y && pipenv sync` remedy. | +| `pypi_hatch_stale_install` | `skipped` (warning) | vendor (hatch): an existing Hatch environment of the project (found under Hatch's data dir, `dirs.env.virtual` or an explicit env `path`) still holds the upstream release; Hatch keeps it on the next `hatch run`, so the detail names the env and the `hatch env remove ` / `hatch env prune` remedy. | | `pypi_pipenv_installer_unknown` | `skipped` (warning) | vendor (pipenv): no `pipenv` answered on PATH; the vendored references assume Pipenv 2018 or later (7–11 cannot consume them — use hosted mode there); `SOCKET_PIPENV_MAJOR` pins the release. | | `vendor_lock_entry_relocked` | revert `warnings[]` | vendor `--revert` / rollback (pipenv): a relock regenerated the wired entry to a registry reference, or removed it; the record is retired (artifact removed, ledger entry dropped) instead of drift-kept. | | `pypi_{poetry,pdm,pipenv}_no_lockfile` | `failed` | vendor (pypi): a lock-less tool marker with no `requirements.txt` fallback — run ` lock`. | diff --git a/crates/socket-patch-cli/src/commands/scan/hosted/python.rs b/crates/socket-patch-cli/src/commands/scan/hosted/python.rs index 8a5445673..7eda12855 100644 --- a/crates/socket-patch-cli/src/commands/scan/hosted/python.rs +++ b/crates/socket-patch-cli/src/commands/scan/hosted/python.rs @@ -44,6 +44,13 @@ pub(super) async fn stale_install_warnings( // fallback to the global interpreters would judge an unrelated Python's // copy of the release (a tool venv on PATH) and warn falsely. // --global / --global-prefix keep their meaning. + // Hatch envs need their own remedy: a reinstall from the rewritten + // pyproject does nothing there (#335). + let hatch_envs = if common.is_global() { + Vec::new() + } else { + socket_patch_core::crawlers::hatch_env::hatch_environments(&common.cwd).await + }; let paths = if common.is_global() { crawler .get_site_packages_paths(&common.crawler_options()) @@ -105,7 +112,11 @@ pub(super) async fn stale_install_warnings( // (`install`, `install --deploy`, `sync` all keep the installed // bytes on every major), and `pipenv uninstall` rewrites the // Pipfile and re-locks the patch away — name the verified remedy. - let remedy = if pipenv_purls.contains(&purl) { + let hatch_env = + socket_patch_core::crawlers::hatch_env::environment_of(&hatch_envs, &site); + let remedy = if let Some(env) = hatch_env { + socket_patch_core::crawlers::hatch_env::stale_install_remedy(&env.name) + } else if pipenv_purls.contains(&purl) { let name = strip_purl_qualifiers(&purl) .strip_prefix("pkg:pypi/") .and_then(|rest| rest.split('@').next()) @@ -214,6 +225,58 @@ mod tests { assert!(out.warnings.is_empty()); } + /// #335: a Hatch env keeps the upstream release after the rewrite, and + /// the probe must find it (Hatch keeps envs out of `./.venv`) and name + /// Hatch's remedy, not "reinstall from the rewritten lock". + #[tokio::test] + async fn hatch_env_gets_the_stale_install_warning_with_hatch_remedy() { + if std::env::var_os("VIRTUAL_ENV").is_some() { + return; // an activated venv takes precedence over project envs + } + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write( + project.join("pyproject.toml"), + "[project]\nname = \"app\"\nversion = \"0.1.0\"\ndependencies = [\"six==1.16.0\"]\n\n[tool.hatch.envs.default]\npath = \"../hatch-envs/app\"\n", + ) + .unwrap(); + let env = tmp.path().join("hatch-envs").join("app"); + std::fs::create_dir_all(&env).unwrap(); + std::fs::write(env.join("pyvenv.cfg"), "home = /usr/bin\n").unwrap(); + let site = if cfg!(windows) { + env.join("Lib").join("site-packages") + } else { + env.join("lib").join("python3.12").join("site-packages") + }; + let dist = site.join("six-1.16.0.dist-info"); + std::fs::create_dir_all(&dist).unwrap(); + std::fs::write(dist.join("METADATA"), "Name: six\nVersion: 1.16.0\n").unwrap(); + std::fs::write(site.join("six.py"), b"upstream").unwrap(); + + let common = crate::args::GlobalArgs { + cwd: project.clone(), + ..Default::default() + }; + let purl = "pkg:pypi/six@1.16.0"; + let confirmed = vec![(purl.to_string(), "six-uuid".to_string())]; + let ledger = BTreeMap::from([("k".into(), record("six-uuid", "six.py", b"patched"))]); + let out = stale_install_warnings(&common, &confirmed, &BTreeSet::new(), &ledger).await; + assert_eq!(out.stale_purls, BTreeSet::from([purl.to_string()])); + assert_eq!(out.warnings.len(), 1); + let detail = out.warnings[0]["detail"].as_str().unwrap(); + assert!(detail.contains("hatch env remove default"), "{detail}"); + assert!( + !detail.contains("Reinstall from the rewritten lock"), + "{detail}" + ); + + // Patched in the env: nothing to warn about. + std::fs::write(site.join("six.py"), b"patched").unwrap(); + let out = stale_install_warnings(&common, &confirmed, &BTreeSet::new(), &ledger).await; + assert!(out.warnings.is_empty()); + } + /// A legacy `.egg-info` install (pip < 23.1 building an sdist without /// `wheel`) is a real copy pip keeps on `install -r`, so the hosted /// stale-install guard must judge it like a `.dist-info` one (#447). diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs index 042001150..f67cae325 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs @@ -579,3 +579,142 @@ fn hatch_toml_environment_dependency_hosted() { fn hatch_toml_environment_dependency_vendored() { flow(Flavor::HatchTomlEnv, Mode::Vendored); } + +/// #335: on a project whose Hatch env ALREADY exists (any developer +/// checkout, a warm CI cache), Hatch keeps the upstream `six` on the next +/// `hatch run`: pip, and uv before Hatch 1.16, never reinstall a present +/// release, and Hatch then records the env as synced. Hatch keeps that env +/// out of tree, so socket-patch must find it on its own (no `VIRTUAL_ENV`): +/// the scan warns with the Hatch remedy and names the env, `vex` from the +/// project root refuses to attest the unpatched env, and the named remedy +/// (`hatch env remove default`) really yields the patched bytes, which vex +/// then attests. +fn existing_env_flow(mode: Mode) { + let Some(hatch) = hatch() else { return }; + let version = hatch.version.clone(); + let what = format!("hatch {version} existing-env {}", mode.label()); + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("proj"); + std::fs::create_dir_all(project.join("app")).unwrap(); + std::fs::write(project.join("app/__init__.py"), "").unwrap(); + write_native(&project, Flavor::Project); + let case_envs = hatch.case_env(&tmp.path().join("case")); + + // The env as a developer has it: created from PyPI, upstream six. + let out = hatch.run(&project, &case_envs, &["env", "create"]); + if !out.status.success() { + skip_or_fail( + REQUIRED, + &format!("{what}: hatch env create: {}", out_text(&out)), + ); + return; + } + let found = hatch.run(&project, &case_envs, &["env", "find"]); + assert_ok(&found, &format!("{what}: hatch env find")); + let env_dir = PathBuf::from( + String::from_utf8_lossy(&found.stdout) + .trim() + .lines() + .last() + .unwrap() + .trim(), + ); + let (_, pristine, marked) = + six_oracle(&venv_bin(&env_dir, "python"), &project).expect("pristine six"); + assert!(!marked, "{what}: the env starts unpatched"); + let patched = [pristine.as_slice(), PATCH_SUFFIX].concat(); + let api = RealApi::start(mode.uuid(), &pristine, &patched); + + // Hatch's own state (data dir, config, HOME) but no VIRTUAL_ENV: the + // crawler has to locate Hatch's out-of-tree env itself. + let (code, env, stderr) = socket_scan(&project, &api, &scan_mode_args(mode), &case_envs); + assert_eq!(code, Some(0), "{what}: scan failed: {env}\n{stderr}"); + let (warnings, code_name) = match mode { + Mode::Hosted => ( + env["redirect"]["warnings"].clone(), + "redirect_pypi_stale_install", + ), + Mode::Vendored => (env["vendor"]["events"].clone(), "pypi_hatch_stale_install"), + }; + let details: Vec = warnings + .as_array() + .into_iter() + .flatten() + .filter(|w| w["code"] == code_name || w["errorCode"] == code_name) + .map(|w| w.to_string()) + .collect(); + assert_eq!( + details.len(), + 1, + "{what}: {code_name} expected: {env}\n{stderr}" + ); + assert!( + details[0].contains("hatch env remove default"), + "{what}: Hatch remedy: {}", + details[0] + ); + + // The next `hatch run` keeps the upstream bytes, as the warning says. + let out = hatch.run(&project, &case_envs, &["run", "python", "-c", "import six"]); + assert_ok(&out, &format!("{what}: hatch run")); + let (_, _, marked) = six_oracle(&venv_bin(&env_dir, "python"), &project).unwrap(); + assert!( + !marked, + "{what}: Hatch reinstalled; the premise no longer holds" + ); + + // vex from the project root sees the Hatch env and attests nothing. + let patch_api = vex_e2e_common::PatchApi::start(vec![( + mode.uuid().to_string(), + view(mode.uuid(), &pristine, &patched), + )]); + let run = vex_e2e_common::VexRun { + patch_server_url: Some(api.uri()), + product: Some(PRODUCT.into()), + envs: case_envs + .iter() + .map(|(k, v)| (k.clone(), v.into())) + .collect(), + ..vex_e2e_common::VexRun::online(&patch_api) + }; + let out = vex_e2e_common::run_vex(&vex_e2e_common::binary(), &project, &run); + vex_e2e_common::assert_absent(out.doc.as_ref(), PURL); + record( + "hatch", + &version, + &format!("existing-env/{}", mode.label()), + "stale-warned", + "pass", + ); + + // The remedy works: the recreated env holds the patch, and vex attests. + let out = hatch.run(&project, &case_envs, &["env", "remove", "default"]); + assert_ok(&out, &format!("{what}: hatch env remove")); + let out = hatch.run(&project, &case_envs, &["run", "python", "-c", "import six"]); + assert_ok(&out, &format!("{what}: hatch run after remove")); + let (_, bytes, marked) = six_oracle(&venv_bin(&env_dir, "python"), &project).unwrap(); + assert!(marked, "{what}: the recreated env must hold the patch"); + assert_eq!(git_sha256(&bytes), git_sha256(&patched), "{what}"); + let out = vex_e2e_common::run_vex(&vex_e2e_common::binary(), &project, &run); + assert_eq!(out.code, Some(0), "{what}: vex after the remedy: {out}"); + assert_attested(out.doc(), PURL, mode.uuid(), mode.marker(), VULNS); + record( + "hatch", + &version, + &format!("existing-env/{}", mode.label()), + "remedy-attested", + "pass", + ); +} + +#[test] +#[ignore = "real Hatch + PyPI; run with --ignored (CI: SOCKET_PATCH_HATCH_E2E_REQUIRED=1)"] +fn hatch_existing_env_hosted() { + existing_env_flow(Mode::Hosted); +} + +#[test] +#[ignore = "real Hatch + PyPI; run with --ignored (CI: SOCKET_PATCH_HATCH_E2E_REQUIRED=1)"] +fn hatch_existing_env_vendored() { + existing_env_flow(Mode::Vendored); +} diff --git a/crates/socket-patch-core/src/crawlers/hatch_env.rs b/crates/socket-patch-core/src/crawlers/hatch_env.rs new file mode 100644 index 000000000..f821e1b1b --- /dev/null +++ b/crates/socket-patch-core/src/crawlers/hatch_env.rs @@ -0,0 +1,672 @@ +//! Where Hatch keeps a project's virtual environments. +//! +//! Hatch never uses `./.venv`: `hatch run` / `hatch shell` / `hatch test` +//! install into envs under its data directory, keyed by the project name and +//! a hash of the project root. Modelled on `hatch/env/virtual.py`, +//! `hatch/cli/application.py::get_env_directory` and +//! `hatch/utils/fs.py::Path.id`, unchanged in layout from Hatch 1.0 through +//! 1.18: +//! +//! - env type directory: `[dirs.env] virtual` from Hatch's config file +//! (absolute, else relative to the project), else +//! `/env/virtual`; the data dir is `HATCH_DATA_DIR`, else +//! `dirs.data` from the config file, else the platform data dir; +//! - an env with an explicit `path` (`[tool.hatch.envs.] path`, +//! `hatch.toml`'s `[envs.] path`, or `HATCH_ENV_TYPE_VIRTUAL_PATH`) +//! lives exactly there; +//! - when the env type directory is `~/.virtualenvs` or inside the project, +//! envs sit flat in it (`/`); +//! - otherwise `///`, where the +//! project name is the PEP 503-normalized `[project] name` (or +//! `-unmanaged` without a `[project]` table), the id is the first 8 +//! chars of the URL-safe base64 sha256 of the project root (casefolded on +//! Windows, and on macOS from Hatch 1.10), and the `default` env is named +//! after the project. + +use std::path::{Path, PathBuf}; + +use toml_edit::{DocumentMut, Item}; + +/// The verified remedy for a Hatch env still holding the upstream release +/// after its dependency was rewired. Hatch only syncs a changed dependency +/// with `pip install` (keeps a same-version release that is already +/// installed) or, before Hatch 1.16, accepts the installed release as +/// satisfying the new reference outright; either way it then records the +/// env as synced and never retries. Only recreating the env helps. +pub fn stale_install_remedy(env_name: &str) -> String { + format!( + "Hatch does not reinstall a release that is already present in an existing \ + environment (its pip installer keeps the installed bytes, uv before Hatch 1.16 \ + skips the sync, and Hatch then records the environment as synced), so the \ + rewired dependency only reaches fresh environments. Recreate this one with \ + `hatch env remove {env_name}` (the next `hatch run` rebuilds it), or run `hatch \ + env prune` for every environment of the project; then `socket-patch vex` \ + re-verifies the installed files." + ) +} + +/// The Hatch env among `envs` whose tree holds `site`. +pub fn environment_of<'e>( + envs: &'e [HatchEnvironment], + site: &Path, +) -> Option<&'e HatchEnvironment> { + envs.iter().find(|env| site.starts_with(&env.prefix)) +} + +/// One Hatch virtual environment of a project, as Hatch names it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct HatchEnvironment { + /// The env name `hatch env remove ` takes (`default` for the + /// env named after the project). + pub name: String, + /// The venv root (the directory holding `pyvenv.cfg`). + pub prefix: PathBuf, +} + +/// The existing Hatch virtual environments of the project at `cwd`. Empty +/// when `cwd` has neither `pyproject.toml` nor `hatch.toml`, or Hatch has +/// created none. +pub async fn hatch_environments(cwd: &Path) -> Vec { + let var = |name: &str| std::env::var(name).ok(); + hatch_environments_with(cwd, &var).await +} + +/// [`hatch_environments`] over an explicit environment. +pub(crate) async fn hatch_environments_with( + cwd: &Path, + var: &impl Fn(&str) -> Option, +) -> Vec { + let pyproject = read_toml(&cwd.join("pyproject.toml")).await; + let hatch_toml = read_toml(&cwd.join("hatch.toml")).await; + if pyproject.is_none() && hatch_toml.is_none() { + return Vec::new(); + } + let root = std::fs::canonicalize(cwd).unwrap_or_else(|_| cwd.to_path_buf()); + let config = read_config(var).await; + let configured = configured_envs(pyproject.as_ref(), hatch_toml.as_ref()); + let project_name = pyproject.as_ref().and_then(project_name); + + let mut found: Vec = Vec::new(); + let mut push = |name: String, prefix: PathBuf| { + if prefix.join("pyvenv.cfg").is_file() && !found.iter().any(|e| e.prefix == prefix) { + found.push(HatchEnvironment { name, prefix }); + } + }; + + // Explicit paths win for the env they name. + let override_path = var("HATCH_ENV_TYPE_VIRTUAL_PATH").filter(|v| !v.trim().is_empty()); + for env in &configured { + if let Some(path) = override_path.as_deref().or(env.path.as_deref()) { + push(env.name.clone(), absolutize(&root, path)); + } + } + if override_path.is_some() && configured.iter().all(|e| e.name != "default") { + push( + "default".to_string(), + absolutize(&root, override_path.as_deref().unwrap()), + ); + } + + let Some(env_dir) = virtual_env_dir(&root, config.as_ref(), var) else { + return found; + }; + let in_project = env_dir.starts_with(&root) + || std::fs::canonicalize(&env_dir).is_ok_and(|d| d.starts_with(&root)); + let shared_flat = home_dir(var).is_some_and(|h| same_path(&env_dir, &h.join(".virtualenvs"))); + + for id in project_ids(&root) { + let name = project_name + .clone() + .unwrap_or_else(|| format!("{id}-unmanaged")); + if in_project { + // A directory inside the project holds only this project's envs. + for (dir_name, prefix) in subdirs(&env_dir) { + push(env_name_for(&dir_name, &name), prefix); + } + break; + } + if shared_flat { + // `~/.virtualenvs` is shared, so only the names this project + // configures are taken from it. + push("default".to_string(), env_dir.join(&name)); + for env in configured.iter().filter(|e| e.name != "default") { + push(env.name.clone(), env_dir.join(&env.name)); + } + break; + } + let storage = env_dir.join(&name).join(&id); + for (dir_name, prefix) in subdirs(&storage) { + push(env_name_for(&dir_name, &name), prefix); + } + } + found +} + +/// `default` is stored under the project's name; every other env under its +/// own. +fn env_name_for(dir_name: &str, project_name: &str) -> String { + if dir_name == project_name { + "default".to_string() + } else { + dir_name.to_string() + } +} + +/// An env Hatch's project config declares, with its explicit `path`. +struct ConfiguredEnv { + name: String, + path: Option, +} + +/// `[tool.hatch.envs.*]` from pyproject, unless hatch.toml carries `envs` +/// (Hatch then reads hatch.toml's `[envs.*]` only). Envs whose `type` is +/// not `virtual` are left out. +fn configured_envs( + pyproject: Option<&DocumentMut>, + hatch_toml: Option<&DocumentMut>, +) -> Vec { + let table = hatch_toml + .and_then(|doc| doc.get("envs")) + .or_else(|| { + pyproject + .and_then(|doc| doc.get("tool")) + .and_then(|tool| tool.get("hatch")) + .and_then(|hatch| hatch.get("envs")) + }) + .and_then(Item::as_table_like); + let Some(table) = table else { + return Vec::new(); + }; + table + .iter() + .filter_map(|(name, item)| { + let env = item.as_table_like()?; + if env + .get("type") + .and_then(Item::as_str) + .is_some_and(|kind| kind != "virtual") + { + return None; + } + Some(ConfiguredEnv { + name: name.to_string(), + path: env + .get("path") + .and_then(Item::as_str) + .filter(|p| !p.is_empty()) + .map(str::to_string), + }) + }) + .collect() +} + +/// The PEP 503-normalized `[project] name` (hatchling's +/// `normalize_project_name`). `None` without a `[project]` table; a table +/// without a name is not a project Hatch can load, so `None` too. +fn project_name(pyproject: &DocumentMut) -> Option { + let name = pyproject.get("project")?.get("name")?.as_str()?; + let mut out = String::with_capacity(name.len()); + let mut in_run = false; + for c in name.chars() { + if matches!(c, '-' | '_' | '.') { + if !in_run { + out.push('-'); + } + in_run = true; + } else { + out.extend(c.to_lowercase()); + in_run = false; + } + } + Some(out) +} + +/// The project ids Hatch may have used for `root`: the hash of the path as +/// written, and of its casefolded form where some Hatch release casefolds +/// (Windows always; macOS from Hatch 1.10). +fn project_ids(root: &Path) -> Vec { + let text = strip_windows_verbatim_prefix(&root.to_string_lossy()); + let mut ids = Vec::new(); + if cfg!(windows) || cfg!(target_os = "macos") { + ids.push(path_id(&text.to_lowercase())); + } + let raw = path_id(&text); + if !ids.contains(&raw) { + ids.push(raw); + } + ids +} + +/// `hatch.utils.fs.Path.id` over an already normalized path string. +fn path_id(text: &str) -> String { + use base64::Engine as _; + use sha2::{Digest, Sha256}; + let digest = Sha256::digest(text.as_bytes()); + let encoded = base64::engine::general_purpose::URL_SAFE.encode(digest); + encoded[..8].to_string() +} + +/// `\\?\C:\x` -> `C:\x`, `\\?\UNC\srv\share` -> `\\srv\share` (Python's +/// `Path.cwd()` never carries the verbatim prefix `canonicalize` adds). +fn strip_windows_verbatim_prefix(text: &str) -> String { + if let Some(rest) = text.strip_prefix(r"\\?\UNC\") { + format!(r"\\{rest}") + } else if let Some(rest) = text.strip_prefix(r"\\?\") { + rest.to_string() + } else { + text.to_string() + } +} + +/// The env type directory for `virtual` (see the module docs). +fn virtual_env_dir( + root: &Path, + config: Option<&DocumentMut>, + var: &impl Fn(&str) -> Option, +) -> Option { + let dirs = config.and_then(|c| c.get("dirs")); + if let Some(configured) = dirs + .and_then(|d| d.get("env")) + .and_then(|e| e.get("virtual")) + .and_then(Item::as_str) + { + return Some(absolutize(root, &expand(configured, var))); + } + let data = var("HATCH_DATA_DIR") + .filter(|v| !v.trim().is_empty()) + .map(|v| expand(&v, var)) + .or_else(|| { + dirs.and_then(|d| d.get("data")) + .and_then(Item::as_str) + .map(|v| expand(v, var)) + }) + .map(PathBuf::from) + .or_else(|| default_data_dir(var))?; + Some(data.join("env").join("virtual")) +} + +/// Hatch's config file: `HATCH_CONFIG`, else `config.toml` in +/// `platformdirs.user_config_dir("hatch")`. +async fn read_config(var: &impl Fn(&str) -> Option) -> Option { + let path = var("HATCH_CONFIG") + .filter(|v| !v.trim().is_empty()) + .map(PathBuf::from) + .or_else(|| default_config_dir(var).map(|d| d.join("config.toml")))?; + read_toml(&path).await +} + +/// `platformdirs.user_data_dir("hatch", appauthor=False)`. +fn default_data_dir(var: &impl Fn(&str) -> Option) -> Option { + if cfg!(windows) { + local_app_data(var).map(|d| d.join("hatch")) + } else if cfg!(target_os = "macos") { + Some( + home_dir(var)? + .join("Library") + .join("Application Support") + .join("hatch"), + ) + } else { + Some( + xdg(var, "XDG_DATA_HOME") + .or_else(|| home_dir(var).map(|h| h.join(".local").join("share")))? + .join("hatch"), + ) + } +} + +/// `platformdirs.user_config_dir("hatch", appauthor=False)`. +fn default_config_dir(var: &impl Fn(&str) -> Option) -> Option { + if cfg!(windows) { + local_app_data(var).map(|d| d.join("hatch")) + } else if cfg!(target_os = "macos") { + Some( + home_dir(var)? + .join("Library") + .join("Application Support") + .join("hatch"), + ) + } else { + Some( + xdg(var, "XDG_CONFIG_HOME") + .or_else(|| home_dir(var).map(|h| h.join(".config")))? + .join("hatch"), + ) + } +} + +fn local_app_data(var: &impl Fn(&str) -> Option) -> Option { + var("LOCALAPPDATA") + .filter(|v| !v.trim().is_empty()) + .map(PathBuf::from) + .or_else(|| home_dir(var).map(|h| h.join("AppData").join("Local"))) +} + +/// An XDG base directory, honoured only when absolute (as platformdirs). +fn xdg(var: &impl Fn(&str) -> Option, name: &str) -> Option { + var(name) + .filter(|v| !v.trim().is_empty()) + .map(PathBuf::from) + .filter(|p| p.is_absolute()) +} + +fn home_dir(var: &impl Fn(&str) -> Option) -> Option { + var("HOME") + .or_else(|| var("USERPROFILE")) + .filter(|v| !v.trim().is_empty()) + .map(PathBuf::from) +} + +/// Hatch's `Path.expand`: `~` and `$VAR` / `${VAR}` (`%VAR%` on Windows +/// is left to the shell that set it and not modelled). +fn expand(text: &str, var: &impl Fn(&str) -> Option) -> String { + let text = match text.strip_prefix('~') { + Some(rest) if rest.is_empty() || rest.starts_with('/') || rest.starts_with('\\') => { + match home_dir(var) { + Some(home) => format!("{}{rest}", home.display()), + None => text.to_string(), + } + } + _ => text.to_string(), + }; + let mut out = String::with_capacity(text.len()); + let mut rest = text.as_str(); + while let Some(at) = rest.find('$') { + out.push_str(&rest[..at]); + let after = &rest[at + 1..]; + let (name, tail) = if let Some(braced) = after.strip_prefix('{') { + match braced.find('}') { + Some(end) => (&braced[..end], &braced[end + 1..]), + None => ("", after), + } + } else { + let end = after + .find(|c: char| !(c.is_ascii_alphanumeric() || c == '_')) + .unwrap_or(after.len()); + (&after[..end], &after[end..]) + }; + match (!name.is_empty()).then(|| var(name)).flatten() { + Some(value) => out.push_str(&value), + None => { + // Python's expandvars leaves an unknown variable as written. + out.push('$'); + out.push_str(&after[..after.len() - tail.len()]); + } + } + rest = tail; + } + out.push_str(rest); + out +} + +/// `path` against the project root, resolved like Hatch's +/// `(root / path).resolve()` when it exists. +fn absolutize(root: &Path, path: &str) -> PathBuf { + let path = PathBuf::from(path); + let joined = if path.is_absolute() { + path + } else { + root.join(path) + }; + std::fs::canonicalize(&joined).unwrap_or(joined) +} + +fn same_path(a: &Path, b: &Path) -> bool { + match (std::fs::canonicalize(a), std::fs::canonicalize(b)) { + (Ok(a), Ok(b)) => a == b, + _ => a == b, + } +} + +/// The subdirectories of `dir` by name (none when it is missing). +fn subdirs(dir: &Path) -> Vec<(String, PathBuf)> { + let Ok(entries) = std::fs::read_dir(dir) else { + return Vec::new(); + }; + let mut out: Vec<(String, PathBuf)> = entries + .flatten() + .filter(|e| e.file_type().is_ok_and(|t| t.is_dir())) + .filter_map(|e| Some((e.file_name().into_string().ok()?, e.path()))) + .collect(); + out.sort(); + out +} + +/// A regular file's TOML (a FIFO or device planted at the path is never +/// opened, so discovery cannot block on it). +async fn read_toml(path: &Path) -> Option { + crate::utils::fs::read_regular_to_string(path) + .await + .ok()? + .parse() + .ok() +} + +#[cfg(test)] +mod tests { + use super::*; + use std::collections::HashMap; + + fn env_of(pairs: &[(&str, String)]) -> impl Fn(&str) -> Option { + let map: HashMap = pairs + .iter() + .map(|(k, v)| (k.to_string(), v.clone())) + .collect(); + move |name: &str| map.get(name).cloned() + } + + fn make_venv(prefix: &Path) { + std::fs::create_dir_all(prefix).unwrap(); + std::fs::write(prefix.join("pyvenv.cfg"), "home = /usr/bin\n").unwrap(); + } + + const PYPROJECT: &str = "[project]\nname = \"My_App.Core\"\nversion = \"0.1.0\"\n"; + + /// Hatch 1.18.1 computes `Path("/home/user/app").id` as `zrSR0Z2A` + /// (`urlsafe_b64encode(sha256(b"/home/user/app").digest())[:8]`). + #[test] + fn path_id_matches_hatch() { + assert_eq!(path_id("/home/user/app"), "zrSR0Z2A"); + } + + #[test] + fn project_name_is_pep503_normalized() { + let doc: DocumentMut = PYPROJECT.parse().unwrap(); + assert_eq!(project_name(&doc).as_deref(), Some("my-app-core")); + } + + #[tokio::test] + async fn finds_default_and_named_envs_under_the_data_dir() { + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write(project.join("pyproject.toml"), PYPROJECT).unwrap(); + let data = tmp.path().join("data"); + let root = std::fs::canonicalize(&project).unwrap(); + let id = project_ids(&root).pop().unwrap(); + let storage = data.join("env/virtual/my-app-core").join(&id); + make_venv(&storage.join("my-app-core")); + make_venv(&storage.join("test")); + // Another project's env with the same name is not ours. + make_venv(&data.join("env/virtual/my-app-core/XXXXXXXX/my-app-core")); + + let var = env_of(&[ + ("HATCH_DATA_DIR", data.display().to_string()), + ("HOME", tmp.path().join("home").display().to_string()), + ]); + let found = hatch_environments_with(&project, &var).await; + assert_eq!( + found, + vec![ + HatchEnvironment { + name: "default".into(), + prefix: storage.join("my-app-core") + }, + HatchEnvironment { + name: "test".into(), + prefix: storage.join("test") + }, + ] + ); + } + + #[tokio::test] + async fn default_data_dir_and_config_dirs_env_virtual() { + let tmp = tempfile::tempdir().unwrap(); + let home = tmp.path().join("home"); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write(project.join("pyproject.toml"), PYPROJECT).unwrap(); + let root = std::fs::canonicalize(&project).unwrap(); + let id = project_ids(&root).pop().unwrap(); + let var = env_of(&[ + ("HOME", home.display().to_string()), + ("LOCALAPPDATA", home.join("lad").display().to_string()), + ]); + let data = default_data_dir(&var).unwrap(); + let prefix = data + .join("env/virtual/my-app-core") + .join(&id) + .join("my-app-core"); + make_venv(&prefix); + let found = hatch_environments_with(&project, &var).await; + assert_eq!(found.len(), 1); + assert_eq!(found[0].prefix, prefix); + + // `[dirs.env] virtual` moves the env type directory. + let config_dir = default_config_dir(&var).unwrap(); + std::fs::create_dir_all(&config_dir).unwrap(); + let elsewhere = tmp.path().join("envs"); + std::fs::write( + config_dir.join("config.toml"), + format!("[dirs.env]\nvirtual = '{}'\n", elsewhere.display()), + ) + .unwrap(); + assert!(hatch_environments_with(&project, &var).await.is_empty()); + let moved = elsewhere.join("my-app-core").join(&id).join("lint"); + make_venv(&moved); + let found = hatch_environments_with(&project, &var).await; + assert_eq!( + found, + vec![HatchEnvironment { + name: "lint".into(), + prefix: moved + }] + ); + } + + #[tokio::test] + async fn in_project_env_dir_is_flat_and_explicit_paths_win() { + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write( + project.join("pyproject.toml"), + format!("{PYPROJECT}\n[tool.hatch.envs.docs]\npath = \"envs/docs\"\n"), + ) + .unwrap(); + let config = tmp.path().join("hatch-config.toml"); + std::fs::write(&config, "[dirs.env]\nvirtual = \".hatch\"\n").unwrap(); + make_venv(&project.join(".hatch/my-app-core")); + make_venv(&project.join(".hatch/test")); + make_venv(&project.join("envs/docs")); + let var = env_of(&[("HATCH_CONFIG", config.display().to_string())]); + let found = hatch_environments_with(&project, &var).await; + let root = std::fs::canonicalize(&project).unwrap(); + let names: Vec<_> = found + .iter() + .map(|e| (e.name.as_str(), e.prefix.clone())) + .collect(); + assert_eq!( + names, + vec![ + ("docs", root.join("envs/docs")), + ("default", root.join(".hatch/my-app-core")), + ("test", root.join(".hatch/test")), + ] + ); + } + + #[tokio::test] + async fn unmanaged_project_and_no_project_files() { + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + let data = tmp.path().join("data"); + let var = env_of(&[("HATCH_DATA_DIR", data.display().to_string())]); + assert!(hatch_environments_with(&project, &var).await.is_empty()); + + std::fs::write(project.join("hatch.toml"), "[envs.default]\n").unwrap(); + let root = std::fs::canonicalize(&project).unwrap(); + let id = project_ids(&root).pop().unwrap(); + let name = format!("{id}-unmanaged"); + let prefix = data.join("env/virtual").join(&name).join(&id).join(&name); + make_venv(&prefix); + let found = hatch_environments_with(&project, &var).await; + assert_eq!( + found, + vec![HatchEnvironment { + name: "default".into(), + prefix + }] + ); + } + + #[cfg(unix)] + #[tokio::test] + async fn fifo_configuration_never_blocks() { + for filename in ["pyproject.toml", "hatch.toml", "config.toml"] { + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + if filename != "pyproject.toml" { + std::fs::write(project.join("pyproject.toml"), PYPROJECT).unwrap(); + } + let fifo = if filename == "config.toml" { + tmp.path().join(filename) + } else { + project.join(filename) + }; + assert!(std::process::Command::new("mkfifo") + .arg(&fifo) + .status() + .unwrap() + .success()); + let var = env_of(&[ + ("HATCH_CONFIG", fifo.display().to_string()), + ( + "HATCH_DATA_DIR", + tmp.path().join("data").display().to_string(), + ), + ]); + let result = tokio::time::timeout( + std::time::Duration::from_secs(2), + hatch_environments_with(&project, &var), + ) + .await; + if result.is_err() { + drop( + std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(&fifo) + .unwrap(), + ); + } + assert!(result.unwrap().is_empty(), "{filename}"); + } + } + + #[test] + fn expand_matches_python() { + let var = env_of(&[("HOME", "/h".to_string()), ("X", "ex".to_string())]); + assert_eq!(expand("~/a/$X/${X}b/$NOPE/c", &var), "/h/a/ex/exb/$NOPE/c"); + assert_eq!(expand("~user/a", &var), "~user/a"); + } +} + +/// [`project_ids`] for tests elsewhere in the crate. +#[cfg(test)] +pub(crate) fn project_ids_for_tests(root: &Path) -> Vec { + project_ids(root) +} diff --git a/crates/socket-patch-core/src/crawlers/mod.rs b/crates/socket-patch-core/src/crawlers/mod.rs index b0c257f50..a983d2d2e 100644 --- a/crates/socket-patch-core/src/crawlers/mod.rs +++ b/crates/socket-patch-core/src/crawlers/mod.rs @@ -3,6 +3,7 @@ pub mod composer_crawler; pub mod deno_crawler; pub mod fuzzy_match; pub mod go_crawler; +pub mod hatch_env; mod listing; pub mod maven_crawler; #[cfg(test)] diff --git a/crates/socket-patch-core/src/crawlers/python_crawler.rs b/crates/socket-patch-core/src/crawlers/python_crawler.rs index dfdee4f52..f9f278cea 100644 --- a/crates/socket-patch-core/src/crawlers/python_crawler.rs +++ b/crates/socket-patch-core/src/crawlers/python_crawler.rs @@ -331,7 +331,9 @@ async fn find_site_packages_under( /// 3. Poetry's out-of-tree virtualenv(s), when Poetry itself would not use /// `./.venv` for the project (see [`find_poetry_virtualenv_site_packages`]) /// 4. `.venv` directory in `cwd` -/// 5. `venv` directory in `cwd` +/// 5. `venv` directory in `cwd` (a PDM project with neither: PEP 582) +/// 6. Hatch's out-of-tree envs for the project (see +/// [`super::hatch_env::hatch_environments`]), added to 4/5 pub async fn find_local_venv_site_packages(cwd: &Path) -> Vec { let var = |name: &str| std::env::var(name).ok(); find_local_venv_site_packages_with(cwd, &var).await @@ -419,6 +421,17 @@ async fn find_local_venv_site_packages_with( results = pdm_pep582_dirs(cwd).await; } + // 6. Hatch never installs into `./.venv` / `./venv`: `hatch run` uses + // its own out-of-tree envs (#335). Every existing one belongs to the + // project, next to whatever a generic probe found. + for env in super::hatch_env::hatch_environments_with(cwd, var).await { + for site in find_site_packages_under(&env.prefix, "site-packages").await { + if !results.contains(&site) { + results.push(site); + } + } + } + results } @@ -2545,6 +2558,54 @@ mod tests { } } + /// #335: Hatch keeps a project's envs out of tree, under + /// `/env/virtual///`, and never uses `./.venv` + /// for them. Every existing env is the project's (stale-install probes, + /// VEX's installed basis and agent mode see them all), alongside a + /// `./.venv` another tool made. + #[tokio::test] + async fn hatch_out_of_tree_envs_are_project_envs() { + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write( + project.join("pyproject.toml"), + "[build-system]\nrequires = [\"hatchling\"]\nbuild-backend = \"hatchling.build\"\n\n[project]\nname = \"app\"\nversion = \"0.1.0\"\ndependencies = [\"six==1.16.0\"]\n", + ) + .unwrap(); + let data = tmp.path().join("hatch-data"); + let var = env_of(&[ + ("HATCH_DATA_DIR", data.to_string_lossy().into_owned()), + ( + "HOME", + tmp.path().join("home").to_string_lossy().into_owned(), + ), + ]); + assert!(find_local_venv_site_packages_with(&project, &var) + .await + .is_empty()); + + let envs = super::super::hatch_env::hatch_environments_with(&project, &var).await; + assert!(envs.is_empty()); + let storage_root = data.join("env").join("virtual").join("app"); + // Hatch's own id for the project root, whichever casefolding applies. + let real = std::fs::canonicalize(&project).unwrap(); + let mut sites = Vec::new(); + for id in super::super::hatch_env::project_ids_for_tests(&real) { + let (_, site) = fake_venv_root(&storage_root.join(&id).join("app")); + sites.push(site); + } + let found = find_local_venv_site_packages_with(&project, &var).await; + assert!(!found.is_empty()); + assert!(found.iter().all(|s| sites.contains(s)), "{found:?}"); + + // A `./.venv` beside them is kept too. + let dot = fake_venv(&project, ".venv"); + let found = find_local_venv_site_packages_with(&project, &var).await; + assert!(found.contains(&dot)); + assert!(found.iter().any(|s| sites.contains(s))); + } + /// #502: PDM installs into the interpreter saved in `.pdm-python` (an /// out-of-tree `venv.in_project = false` venv, or one bound with /// `pdm use`), ahead of a stray `./.venv` and an activated venv. diff --git a/crates/socket-patch-core/src/vendor/pypi.rs b/crates/socket-patch-core/src/vendor/pypi.rs index 74883c23e..b4c7b6f17 100644 --- a/crates/socket-patch-core/src/vendor/pypi.rs +++ b/crates/socket-patch-core/src/vendor/pypi.rs @@ -609,10 +609,81 @@ async fn pipenv_stale_install_warning( record: &PatchRecord, listings: &InstalledSiteListings, ) -> Option { - use crate::crawlers::python_crawler::{find_local_venv_site_packages, PythonCrawler}; + use crate::crawlers::python_crawler::find_local_venv_site_packages; + let sites = find_local_venv_site_packages(project_root).await; + let stale_dirs = stale_install_sites(&sites, purl, record, listings).await; + if stale_dirs.is_empty() { + return None; + } + let name = parse_pypi_purl(strip_purl_qualifiers(purl)) + .map(|(name, _)| name.to_string()) + .unwrap_or_else(|| purl.to_string()); + let listed = stale_dirs + .iter() + .map(|d| d.display().to_string()) + .collect::>() + .join(", "); + Some(VendorWarning::new( + "pypi_pipenv_stale_install", + format!( + "{purl}: the UNPATCHED upstream release is still installed in {listed}. Pipenv does not reinstall a release that is already present (`pipenv install`, `pipenv install --deploy` and `pipenv sync` all keep those bytes), so the wired Pipfile.lock only protects fresh installs. Reinstall it from the lock without touching the Pipfile: `pipenv run pip uninstall -y {name} && pipenv sync` (`pipenv install --deploy` before Pipenv 2018), or `pipenv --rm && pipenv sync` for a clean virtualenv — NOT `pipenv uninstall`, which rewrites the Pipfile and re-locks the patch away; then `socket-patch vex` re-verifies the installed files." + ), + )) +} + +/// The Hatch twin of [`pipenv_stale_install_warning`]: Hatch keeps the +/// upstream release installed in an existing env (#335), so each env that +/// still holds it gets `pypi_hatch_stale_install` with the env-recreating +/// remedy. Only Hatch's own envs are judged (an activated one included): a +/// `./.venv` another tool made is not where `hatch run` installs. +async fn hatch_stale_install_warning( + project_root: &Path, + purl: &str, + record: &PatchRecord, + listings: &InstalledSiteListings, +) -> Vec { + use crate::crawlers::hatch_env::{environment_of, hatch_environments, stale_install_remedy}; + use crate::crawlers::python_crawler::find_local_venv_site_packages; + let envs = hatch_environments(project_root).await; + if envs.is_empty() { + return Vec::new(); + } + let sites: Vec = find_local_venv_site_packages(project_root) + .await + .into_iter() + .filter(|site| environment_of(&envs, site).is_some()) + .collect(); + stale_install_sites(&sites, purl, record, listings) + .await + .into_iter() + .filter_map(|site| { + let env = environment_of(&envs, &site)?; + Some(VendorWarning::new( + "pypi_hatch_stale_install", + format!( + "{purl}: the UNPATCHED upstream release is still installed in the Hatch environment `{}` ({}). {}", + env.name, + site.display(), + stale_install_remedy(&env.name) + ), + )) + }) + .collect() +} + +/// The `sites` that hold the package of `purl` with positive evidence of +/// unpatched bytes (a readable file at the upstream or another hash), and no +/// copy that verifies as patched. +async fn stale_install_sites( + sites: &[std::path::PathBuf], + purl: &str, + record: &PatchRecord, + listings: &InstalledSiteListings, +) -> Vec { + use crate::crawlers::python_crawler::PythonCrawler; use crate::patch::apply::{verify_file_patch, VerifyStatus}; if record.files.is_empty() { - return None; + return Vec::new(); } // Judged over the PROJECT'S venvs (VIRTUAL_ENV, ./.venv, ./venv, Pipenv's // WORKON_HOME venv) — never the staging dir a lock-only vendor fetched @@ -622,20 +693,20 @@ async fn pipenv_stale_install_warning( let base = strip_purl_qualifiers(purl).to_string(); let crawler = PythonCrawler::new(); let mut stale_dirs: Vec = Vec::new(); - for site in find_local_venv_site_packages(project_root).await { - let listed = listings.of(&site).await; - let found = crawler.find_by_purls_listed(&site, &listed, std::slice::from_ref(&base)); + for site in sites { + let listed = listings.of(site).await; + let found = crawler.find_by_purls_listed(site, &listed, std::slice::from_ref(&base)); if !found.contains_key(&base) { continue; } - if crate::vex::verify::verify_patch_record(&site, record) + if crate::vex::verify::verify_patch_record(site, record) .await .is_ok() { continue; } for (file, info) in &record.files { - let result = verify_file_patch(&site, file, info).await; + let result = verify_file_patch(site, file, info).await; if matches!( result.status, VerifyStatus::Ready | VerifyStatus::HashMismatch @@ -646,23 +717,7 @@ async fn pipenv_stale_install_warning( } } } - if stale_dirs.is_empty() { - return None; - } - let name = parse_pypi_purl(strip_purl_qualifiers(purl)) - .map(|(name, _)| name.to_string()) - .unwrap_or_else(|| purl.to_string()); - let listed = stale_dirs - .iter() - .map(|d| d.display().to_string()) - .collect::>() - .join(", "); - Some(VendorWarning::new( - "pypi_pipenv_stale_install", - format!( - "{purl}: the UNPATCHED upstream release is still installed in {listed}. Pipenv does not reinstall a release that is already present (`pipenv install`, `pipenv install --deploy` and `pipenv sync` all keep those bytes), so the wired Pipfile.lock only protects fresh installs. Reinstall it from the lock without touching the Pipfile: `pipenv run pip uninstall -y {name} && pipenv sync` (`pipenv install --deploy` before Pipenv 2018), or `pipenv --rm && pipenv sync` for a clean virtualenv — NOT `pipenv uninstall`, which rewrites the Pipfile and re-locks the patch away; then `socket-patch vex` re-verifies the installed files." - ), - )) + stale_dirs } /// Everything [`vendor_pypi_with_pipenv_version`] decides before it can @@ -780,7 +835,17 @@ async fn pypi_prelude<'p>( } } PypiFlavor::Hatch => { - match super::pypi_hatch::load(project_root, &canon_name, version, &record.uuid).await { + let loaded = + super::pypi_hatch::load(project_root, &canon_name, version, &record.uuid).await; + // Both a fresh vendor and a re-run over already-wired + // dependencies keep warning while a Hatch env still holds the + // upstream release (#335). A refusal probes nothing. + if loaded.is_ok() { + warnings.extend( + hatch_stale_install_warning(project_root, purl, record, installed_sites).await, + ); + } + match loaded { Ok(project) if project.in_sync => { wired_pin = project.pin; WiringPlan::InSync @@ -4990,6 +5055,88 @@ wheels = [ } } + /// #335: Hatch keeps the upstream release in an existing env (pip, and + /// uv before Hatch 1.16, never reinstall it), so vendoring a Hatch + /// project — fresh and re-run in sync — must name each such env with + /// the `hatch env remove` remedy. A `./.venv` Hatch does not use stays + /// out of it; a patched env gets nothing. + #[tokio::test] + async fn hatch_vendor_warns_about_a_stale_hatch_env() { + if std::env::var_os("VIRTUAL_ENV").is_some() { + return; // an activated venv takes precedence over project envs + } + let fx = e2e_fixture().await; + swap_to_lock_flavor( + &fx, + &[( + "pyproject.toml", + "[build-system]\nrequires = [\"hatchling\"]\nbuild-backend = \"hatchling.build\"\n\n[project]\nname = \"proj\"\nversion = \"0.1.0\"\ndependencies = [\"six==1.16.0\"]\n\n[tool.hatch.envs.default]\npath = \".hatch-env\"\n", + )], + ) + .await; + let env = fx.root.join(".hatch-env"); + let site = if cfg!(windows) { + env.join("Lib").join("site-packages") + } else { + env.join("lib").join("python3.12").join("site-packages") + }; + tokio::fs::create_dir_all(site.join("six-1.16.0.dist-info")) + .await + .unwrap(); + touch(&env, "pyvenv.cfg", "home = /usr/bin\n").await; + touch(&site, "six.py", std::str::from_utf8(ORIG).unwrap()).await; + touch( + &site.join("six-1.16.0.dist-info"), + "METADATA", + "Metadata-Version: 2.1\nName: six\nVersion: 1.16.0\n", + ) + .await; + + let stale = |warnings: &[VendorWarning]| -> Vec { + warnings + .iter() + .filter(|w| w.code == "pypi_hatch_stale_install") + .map(|w| w.detail.clone()) + .collect() + }; + let VendorOutcome::Done { + result, + entry, + warnings, + } = vendor_six(&fx, &PatchSources::blobs_only(&fx.blobs), None).await + else { + panic!("vendor must be Done"); + }; + assert!(result.success, "{:?}", result.error); + crate::vendor::test_support::persist(&fx.root, "pkg:pypi/six@1.16.0", entry.unwrap()).await; + let details = stale(&warnings); + assert_eq!(details.len(), 1, "{warnings:?}"); + assert!( + details[0].contains("hatch env remove default"), + "{}", + details[0] + ); + assert!(details[0].contains("six.py") || details[0].contains("site-packages")); + assert!(!details[0].contains(".venv"), "{}", details[0]); + + // The in-sync re-run keeps warning while the env is stale. + let VendorOutcome::Done { warnings, .. } = + vendor_six(&fx, &PatchSources::blobs_only(&fx.blobs), None).await + else { + panic!("re-run must be Done"); + }; + assert_eq!(stale(&warnings).len(), 1, "{warnings:?}"); + + // Once the env holds the patched bytes there is nothing to say. + touch(&site, "six.py", std::str::from_utf8(PATCHED).unwrap()).await; + let VendorOutcome::Done { warnings, .. } = + vendor_six(&fx, &PatchSources::blobs_only(&fx.blobs), None).await + else { + panic!("re-run must be Done"); + }; + assert!(stale(&warnings).is_empty(), "{warnings:?}"); + } + /// One full vendor → revert cycle through `vendor_pypi` for a lock-splice /// flavor: plan arm, wire arm, `entry.flavor` tag (PypiFlavor::as_str), /// the matching MetaSlot, and the byte-identical lock restore. From f79035b6f79b51435d3ce5833a04bd6d9de0eb21 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 14:59:45 +0000 Subject: [PATCH 2/5] Name Hatch's remedy in vendored vex and old layouts Hatch 1.0 to 1.2 keep envs at -/, so discover that layout too. Vendored vex already warns when the installed tree is out of sync with the committed artifact; for a Hatch project the advice to re-run the install does nothing, so name hatch env remove instead. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 --- crates/socket-patch-cli/CLI_CONTRACT.md | 2 +- crates/socket-patch-cli/src/commands/vex.rs | 22 +++++++++++- .../tests/e2e_vex_build/hatch.rs | 22 ++++++++++-- .../src/crawlers/hatch_env.rs | 36 ++++++++++++++++++- 4 files changed, 77 insertions(+), 5 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 6e8c61b79..7e642078d 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -327,7 +327,7 @@ Contract details: * **JSON success surface**: `apply` adds a top-level `vex` object to its envelope; `scan` adds a top-level `vex` key to its result. Both carry `{ path, statements, format: "openvex-0.2.0" }`. * `apply`'s no-manifest early exit (the `noManifest` success no-op; v5.0: its human line is `No patch manifest found; nothing to apply.` — it names the missing `.socket/manifest.json`, not the folder, since `.socket/` may legitimately hold vendored state) and `vendor`'s (`No manifest found, nothing to vendor.` — a project with hosted pins ejects instead, v5.0) still generate the document from the lockfiles and the vendor ledger (manifest-less VEX: hosted / vendored checkouts carry no manifest). Nothing referenced anywhere keeps the calm exit 0 (a stale document at the path is removed; `--json` carries any discovery diagnostics in `warnings[]`); any other VEX failure fails the command with exit 1 — including a run whose only candidates are omitted `record_unavailable` (an `--offline` run over a lockfile-wired checkout with no local records), so an ambient `SOCKET_VEX` there fails the install. `--dry-run` skips generation on both, and so does `apply --check` — it stays read-only and offline-safe, leaving the output path untouched. `scan` has no such early exit: with no manifest and nothing wired anywhere its `--vex` fails with `manifest_not_found`. * **Stale-doc removal (v3.5)**: a run that ends in a VEX error removes a recognizably-OpenVEX file (JSON whose `@context` names openvex.dev) already sitting at the output path — a pipeline reusing one path can never ship yesterday's attestation for a now-unpatched tree. Unrelated files at the path are never touched; a mid-write partial that no longer parses as JSON is left for downstream parsers to reject loudly. -* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install; the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the authenticated API refused the credentials and the public proxy served free patches only; `get` / `scan`'s warning text) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). +* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the authenticated API refused the credentials and the public proxy served free patches only; `get` / `scan`'s warning text) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). ### VEX provenance markers (contract) diff --git a/crates/socket-patch-cli/src/commands/vex.rs b/crates/socket-patch-cli/src/commands/vex.rs index 43eff8991..6d5615806 100644 --- a/crates/socket-patch-cli/src/commands/vex.rs +++ b/crates/socket-patch-cli/src/commands/vex.rs @@ -674,6 +674,26 @@ async fn generate_vex( // installed tree is present and running different bytes. Say so — a // build that bypasses the vendor wiring is unpatched until the next // package-manager install. + // A Hatch env is never resynced by an install: Hatch keeps a present + // release (#335), so name the remedy that recreates it. + let hatch_note = if outcome.vendored_out_of_sync.is_empty() || common.is_global() { + String::new() + } else { + match socket_patch_core::crawlers::hatch_env::hatch_environments(&common.cwd) + .await + .as_slice() + { + [] => String::new(), + envs => format!( + " A Hatch environment keeps an installed release on the next `hatch run`; \ + recreate it instead ({}).", + envs.iter() + .map(|env| format!("`hatch env remove {}`", env.name)) + .collect::>() + .join(", ") + ), + } + }; for purl in &outcome.vendored_out_of_sync { note_warning( warnings, @@ -683,7 +703,7 @@ async fn generate_vex( "{purl}: the installed tree does not match its vendored artifact; the \ attestation is based on the committed .socket/vendor artifact (the lockfile \ consumes it), but the live tree carries different bytes — re-run your \ - package manager's install to resync it." + package manager's install to resync it.{hatch_note}" ), ); } diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs index f67cae325..1a699df42 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs @@ -663,7 +663,9 @@ fn existing_env_flow(mode: Mode) { "{what}: Hatch reinstalled; the premise no longer holds" ); - // vex from the project root sees the Hatch env and attests nothing. + // vex from the project root sees the Hatch env: hosted attests + // nothing over it; vendored attests the committed artifact (its + // contract) but discloses the out-of-sync env with Hatch's remedy. let patch_api = vex_e2e_common::PatchApi::start(vec![( mode.uuid().to_string(), view(mode.uuid(), &pristine, &patched), @@ -678,7 +680,23 @@ fn existing_env_flow(mode: Mode) { ..vex_e2e_common::VexRun::online(&patch_api) }; let out = vex_e2e_common::run_vex(&vex_e2e_common::binary(), &project, &run); - vex_e2e_common::assert_absent(out.doc.as_ref(), PURL); + match mode { + Mode::Hosted => vex_e2e_common::assert_absent(out.doc.as_ref(), PURL), + Mode::Vendored => { + assert_attested(out.doc(), PURL, mode.uuid(), mode.marker(), VULNS); + let disclosed = out.envelope["warnings"] + .as_array() + .into_iter() + .flatten() + .any(|w| { + w["code"] == "vendored_tree_out_of_sync" + && w["detail"] + .as_str() + .is_some_and(|d| d.contains("hatch env remove default")) + }); + assert!(disclosed, "{what}: vendored_tree_out_of_sync with Hatch remedy: {out}"); + } + } record( "hatch", &version, diff --git a/crates/socket-patch-core/src/crawlers/hatch_env.rs b/crates/socket-patch-core/src/crawlers/hatch_env.rs index f821e1b1b..6d6ce1f81 100644 --- a/crates/socket-patch-core/src/crawlers/hatch_env.rs +++ b/crates/socket-patch-core/src/crawlers/hatch_env.rs @@ -5,7 +5,7 @@ //! a hash of the project root. Modelled on `hatch/env/virtual.py`, //! `hatch/cli/application.py::get_env_directory` and //! `hatch/utils/fs.py::Path.id`, unchanged in layout from Hatch 1.0 through -//! 1.18: +//! 1.18 except where noted: //! //! - env type directory: `[dirs.env] virtual` from Hatch's config file //! (absolute, else relative to the project), else @@ -16,6 +16,8 @@ //! lives exactly there; //! - when the env type directory is `~/.virtualenvs` or inside the project, //! envs sit flat in it (`/`); +//! - Hatch 1.0 - 1.2 keep every env at `/-/`; //! - otherwise `///`, where the //! project name is the PEP 503-normalized `[project] name` (or //! `-unmanaged` without a `[project]` table), the id is the first 8 @@ -115,6 +117,13 @@ pub(crate) async fn hatch_environments_with( let shared_flat = home_dir(var).is_some_and(|h| same_path(&env_dir, &h.join(".virtualenvs"))); for id in project_ids(&root) { + // Hatch 1.0 - 1.2: `/-/`, whatever + // the directory (no flat or unmanaged layouts yet). + if let Some(name) = &project_name { + for (dir_name, prefix) in subdirs(&env_dir.join(format!("{name}-{id}"))) { + push(env_name_for(&dir_name, name), prefix); + } + } let name = project_name .clone() .unwrap_or_else(|| format!("{id}-unmanaged")); @@ -510,6 +519,31 @@ mod tests { ); } + /// Hatch 1.0 - 1.2 (`hatch/env/virtual.py` there): `-`. + #[tokio::test] + async fn legacy_hatch_layout() { + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write(project.join("pyproject.toml"), PYPROJECT).unwrap(); + let data = tmp.path().join("data"); + let root = std::fs::canonicalize(&project).unwrap(); + let id = project_ids(&root).pop().unwrap(); + let prefix = data + .join("env/virtual") + .join(format!("my-app-core-{id}")) + .join("my-app-core"); + make_venv(&prefix); + let var = env_of(&[("HATCH_DATA_DIR", data.display().to_string())]); + assert_eq!( + hatch_environments_with(&project, &var).await, + vec![HatchEnvironment { + name: "default".into(), + prefix + }] + ); + } + #[tokio::test] async fn default_data_dir_and_config_dirs_env_virtual() { let tmp = tempfile::tempdir().unwrap(); From 2621cc4bc8685ad7dcf8ff8160864d37ed45bdd1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 15:00:49 +0000 Subject: [PATCH 3/5] Document Hatch existing-env handling Explain where Hatch keeps environments, which warning each mode gives for a stale one and the remedy, and how to run the real-Hatch check. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 --- .../socket-patch-cli/tests/e2e_vex_build/hatch.rs | 5 ++++- docs/testing/hatch.md | 13 +++++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs index 1a699df42..b00e56480 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs @@ -694,7 +694,10 @@ fn existing_env_flow(mode: Mode) { .as_str() .is_some_and(|d| d.contains("hatch env remove default")) }); - assert!(disclosed, "{what}: vendored_tree_out_of_sync with Hatch remedy: {out}"); + assert!( + disclosed, + "{what}: vendored_tree_out_of_sync with Hatch remedy: {out}" + ); } } record( diff --git a/docs/testing/hatch.md b/docs/testing/hatch.md index fa1742976..543db2867 100644 --- a/docs/testing/hatch.md +++ b/docs/testing/hatch.md @@ -33,10 +33,23 @@ the direct-reference permission. Selective and preserved rollback retain the set project direct reference remains, and restore its original value after the last reference is unwired. +Existing environments: Hatch keeps a project's environments out of tree +(`/env/virtual///`, or `HATCH_DATA_DIR`, +`[dirs.env] virtual`, an env's `path`; Hatch 1.0-1.2 use +`-/`), and on the next `hatch run` its pip installer (and +uv before Hatch 1.16) keeps the release already installed there. Socket +Patch finds those environments itself: a hosted scan warns +`redirect_pypi_stale_install`, a vendored one `pypi_hatch_stale_install`, +both naming `hatch env remove ` / `hatch env prune`; hosted `vex` +does not attest over a stale env, and vendored `vex` discloses it with +`vendored_tree_out_of_sync`. Agent mode crawls the same environments. + Focused Rust checks: ```sh cargo test --locked -p socket-patch-core --lib hatch +# real Hatch, existing envs (needs uv + PyPI): +SOCKET_PATCH_HATCH_E2E_REQUIRED=1 cargo test -p socket-patch-cli --test e2e_vex_build -- hatch::hatch_existing_env --ignored ``` The depscan companion PR runs real released Hatch binaries, actual CLI From f8905e1a02aff2b0013cece42d385510b0c1658d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 15:23:48 +0000 Subject: [PATCH 4/5] Match Hatch envs across symlinked path spellings On macOS /var is a symlink to /private/var, so an activated env and the discovered one can name the same directory differently, and the stale-install check then missed it. Compare resolved paths as a fallback, and leave dirs.env.virtual unresolved as Hatch does (only an env's explicit path is resolved). Refs #335 Assisted-by: Claude Code:claude-opus-5-5 --- .../src/crawlers/hatch_env.rs | 48 +++++++++++++++---- 1 file changed, 40 insertions(+), 8 deletions(-) diff --git a/crates/socket-patch-core/src/crawlers/hatch_env.rs b/crates/socket-patch-core/src/crawlers/hatch_env.rs index 6d6ce1f81..d2bb431f6 100644 --- a/crates/socket-patch-core/src/crawlers/hatch_env.rs +++ b/crates/socket-patch-core/src/crawlers/hatch_env.rs @@ -47,12 +47,19 @@ pub fn stale_install_remedy(env_name: &str) -> String { ) } -/// The Hatch env among `envs` whose tree holds `site`. +/// The Hatch env among `envs` whose tree holds `site`, however either is +/// spelled (an activated `VIRTUAL_ENV` may name the env through a symlink, +/// e.g. macOS's `/var` -> `/private/var`). pub fn environment_of<'e>( envs: &'e [HatchEnvironment], site: &Path, ) -> Option<&'e HatchEnvironment> { - envs.iter().find(|env| site.starts_with(&env.prefix)) + if let Some(env) = envs.iter().find(|env| site.starts_with(&env.prefix)) { + return Some(env); + } + let site = std::fs::canonicalize(site).ok()?; + envs.iter() + .find(|env| std::fs::canonicalize(&env.prefix).is_ok_and(|prefix| site.starts_with(prefix))) } /// One Hatch virtual environment of a project, as Hatch names it. @@ -99,13 +106,13 @@ pub(crate) async fn hatch_environments_with( let override_path = var("HATCH_ENV_TYPE_VIRTUAL_PATH").filter(|v| !v.trim().is_empty()); for env in &configured { if let Some(path) = override_path.as_deref().or(env.path.as_deref()) { - push(env.name.clone(), absolutize(&root, path)); + push(env.name.clone(), resolve_env_path(&root, path)); } } if override_path.is_some() && configured.iter().all(|e| e.name != "default") { push( "default".to_string(), - absolutize(&root, override_path.as_deref().unwrap()), + resolve_env_path(&root, override_path.as_deref().unwrap()), ); } @@ -408,15 +415,19 @@ fn expand(text: &str, var: &impl Fn(&str) -> Option) -> String { out } -/// `path` against the project root, resolved like Hatch's -/// `(root / path).resolve()` when it exists. +/// `path` against the project root, as Hatch joins it. fn absolutize(root: &Path, path: &str) -> PathBuf { let path = PathBuf::from(path); - let joined = if path.is_absolute() { + if path.is_absolute() { path } else { root.join(path) - }; + } +} + +/// An env's explicit `path`, resolved like Hatch's `(root / path).resolve()`. +fn resolve_env_path(root: &Path, path: &str) -> PathBuf { + let joined = absolutize(root, path); std::fs::canonicalize(&joined).unwrap_or(joined) } @@ -691,6 +702,27 @@ mod tests { } } + /// macOS spells temp dirs `/var/...` and `/private/var/...`: an + /// activated env named through a symlink is still that Hatch env. + #[cfg(unix)] + #[test] + fn environment_of_sees_through_symlinked_spellings() { + let tmp = tempfile::tempdir().unwrap(); + let real = tmp.path().join("real"); + let site = real.join("env/lib/python3.12/site-packages"); + std::fs::create_dir_all(&site).unwrap(); + let link = tmp.path().join("link"); + std::os::unix::fs::symlink(&real, &link).unwrap(); + let envs = vec![HatchEnvironment { + name: "default".into(), + prefix: real.join("env"), + }]; + let via_link = link.join("env/lib/python3.12/site-packages"); + assert_eq!(environment_of(&envs, &via_link).unwrap().name, "default"); + assert_eq!(environment_of(&envs, &site).unwrap().name, "default"); + assert!(environment_of(&envs, tmp.path()).is_none()); + } + #[test] fn expand_matches_python() { let var = env_of(&[("HOME", "/h".to_string()), ("X", "ex".to_string())]); From 04610c192b9d7c292ef02ad4848932adec784877 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 15:35:59 +0000 Subject: [PATCH 5/5] Keep Hatch envs visible under an activated venv An activated venv that is not one of Hatch's own is never used by hatch run, yet it returned from discovery before the Hatch envs were added, so a developer shell with any venv active hid the stale Hatch env again. Add Hatch's envs in that case too, and have the vendored probe judge Hatch's env prefixes directly. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 --- .../src/commands/scan/hosted/python.rs | 3 -- .../src/crawlers/python_crawler.rs | 43 ++++++++++++++++++- crates/socket-patch-core/src/vendor/pypi.rs | 15 +++---- 3 files changed, 47 insertions(+), 14 deletions(-) diff --git a/crates/socket-patch-cli/src/commands/scan/hosted/python.rs b/crates/socket-patch-cli/src/commands/scan/hosted/python.rs index 7eda12855..77e27a9ae 100644 --- a/crates/socket-patch-cli/src/commands/scan/hosted/python.rs +++ b/crates/socket-patch-cli/src/commands/scan/hosted/python.rs @@ -230,9 +230,6 @@ mod tests { /// Hatch's remedy, not "reinstall from the rewritten lock". #[tokio::test] async fn hatch_env_gets_the_stale_install_warning_with_hatch_remedy() { - if std::env::var_os("VIRTUAL_ENV").is_some() { - return; // an activated venv takes precedence over project envs - } let tmp = tempfile::tempdir().unwrap(); let project = tmp.path().join("app"); std::fs::create_dir_all(&project).unwrap(); diff --git a/crates/socket-patch-core/src/crawlers/python_crawler.rs b/crates/socket-patch-core/src/crawlers/python_crawler.rs index f9f278cea..8d00bf5ce 100644 --- a/crates/socket-patch-core/src/crawlers/python_crawler.rs +++ b/crates/socket-patch-core/src/crawlers/python_crawler.rs @@ -380,6 +380,9 @@ async fn find_local_venv_site_packages_with( let matches = find_site_packages_under(&venv_path, "site-packages").await; results.extend(matches); if !results.is_empty() { + // `hatch run` / `hatch shell` never use an activated venv that + // is not one of Hatch's own (#335): its envs stay the project's. + add_hatch_site_packages(cwd, var, &mut results).await; return results; } } @@ -424,15 +427,31 @@ async fn find_local_venv_site_packages_with( // 6. Hatch never installs into `./.venv` / `./venv`: `hatch run` uses // its own out-of-tree envs (#335). Every existing one belongs to the // project, next to whatever a generic probe found. + add_hatch_site_packages(cwd, var, &mut results).await; + + results +} + +/// Appends the `site-packages` of every existing Hatch env of the project +/// at `cwd` (see [`super::hatch_env::hatch_environments`]) not already in +/// `results`. +async fn add_hatch_site_packages( + cwd: &Path, + var: &impl Fn(&str) -> Option, + results: &mut Vec, +) { for env in super::hatch_env::hatch_environments_with(cwd, var).await { - for site in find_site_packages_under(&env.prefix, "site-packages").await { + for site in hatch_env_site_packages(&env).await { if !results.contains(&site) { results.push(site); } } } +} - results +/// The `site-packages` directories of one Hatch env. +pub async fn hatch_env_site_packages(env: &super::hatch_env::HatchEnvironment) -> Vec { + find_site_packages_under(&env.prefix, "site-packages").await } /// The `site-packages` of the env the project's package manager records for @@ -2599,6 +2618,26 @@ mod tests { assert!(!found.is_empty()); assert!(found.iter().all(|s| sites.contains(s)), "{found:?}"); + // An activated venv that is not Hatch's does not hide them. + let other = tempfile::tempdir().unwrap(); + let activated_site = fake_venv(other.path(), "tool-venv"); + let mut activated = vec![( + "VIRTUAL_ENV", + other + .path() + .join("tool-venv") + .to_string_lossy() + .into_owned(), + )]; + activated.push(("HATCH_DATA_DIR", data.to_string_lossy().into_owned())); + activated.push(( + "HOME", + tmp.path().join("home").to_string_lossy().into_owned(), + )); + let found = find_local_venv_site_packages_with(&project, &env_of(&activated)).await; + assert_eq!(found.first(), Some(&activated_site), "{found:?}"); + assert!(found.iter().any(|s| sites.contains(s)), "{found:?}"); + // A `./.venv` beside them is kept too. let dot = fake_venv(&project, ".venv"); let found = find_local_venv_site_packages_with(&project, &var).await; diff --git a/crates/socket-patch-core/src/vendor/pypi.rs b/crates/socket-patch-core/src/vendor/pypi.rs index b4c7b6f17..8f7fe84f0 100644 --- a/crates/socket-patch-core/src/vendor/pypi.rs +++ b/crates/socket-patch-core/src/vendor/pypi.rs @@ -643,16 +643,16 @@ async fn hatch_stale_install_warning( listings: &InstalledSiteListings, ) -> Vec { use crate::crawlers::hatch_env::{environment_of, hatch_environments, stale_install_remedy}; - use crate::crawlers::python_crawler::find_local_venv_site_packages; let envs = hatch_environments(project_root).await; if envs.is_empty() { return Vec::new(); } - let sites: Vec = find_local_venv_site_packages(project_root) - .await - .into_iter() - .filter(|site| environment_of(&envs, site).is_some()) - .collect(); + // Judged over Hatch's own envs only (an activated one is among them): a + // `./.venv` another tool made is not where `hatch run` installs. + let mut sites: Vec = Vec::new(); + for env in &envs { + sites.extend(crate::crawlers::python_crawler::hatch_env_site_packages(env).await); + } stale_install_sites(&sites, purl, record, listings) .await .into_iter() @@ -5062,9 +5062,6 @@ wheels = [ /// out of it; a patched env gets nothing. #[tokio::test] async fn hatch_vendor_warns_about_a_stale_hatch_env() { - if std::env::var_os("VIRTUAL_ENV").is_some() { - return; // an activated venv takes precedence over project envs - } let fx = e2e_fixture().await; swap_to_lock_flavor( &fx,