Skip to content

Share one path normalizer, home resolver and repository-root walk - #1038

Open
Mikola Lysenko (mikolalysenko) wants to merge 9 commits into
mainfrom
arch-fix/paths-roots
Open

Mikola Lysenko (mikolalysenko) wants to merge 9 commits into
mainfrom
arch-fix/paths-roots

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The architecture audit (§3.B, "Identity, equality and liveness checks") found the same path logic written many times, with rules that drift apart:

  • Path normalizers. Twelve hand-rolled ./.. folding loops, each with its own rules (fail on escape, keep the escape, or ignore ..; one separator or both; a floor or none). One of them had a real bug: the pnpm crawler's copy let a second leading .. pop the first, because PathBuf::pop also removes a .. segment. So ../../x collapsed to x.
  • Repository-root walks (B22). Three ancestor walks each decided which repository a directory belongs to:
    • policy::find_repo_root_with_warnings (socket.yml) honored .git files, the home-directory stop, GIT_CEILING_DIRECTORIES and owner trust.
    • VEX product detection (vex/product.rs find_git_config) only looked for <dir>/.git/config and walked all the way to /. Inside a submodule it named the superproject as the product; in a linked worktree it found no remote at all; a dotfiles repository at $HOME became the product of every project under it.
    • The JVM not_build_root check stopped at any .git, but crossed the home directory and the ceilings.
  • B66. Home resolvers fell back to a relative path. utils::fs::home_dir returned the literal ~, and every caller (~/.cargo registry and config chain, nuget, deno, ruby, python user site-packages, pipx/uv/pdm/conda/pyenv roots) joined onto it, so the probes resolved against the working directory. The Maven default (m2_repo_path_with) had its own copy of the same fallback, so with no home the Maven local repository was ./~/.m2/repository inside the scanned project. The Python crawler's injected-env readers took an empty or relative HOME as-is.
  • B74. get read pnpm-lock.yaml with a plain read_to_string. cargo_workspace and the maven checksum-sidecar reader did an lstat, then a blocking read: a hand-rolled copy of the regular-file guard with a check-then-open race.

No GitHub issue tracks these. They are audit IDs B22, B66 and B74, plus the §3.B duplication rows.

Change

  1. utils::relpath. One segment stack (Segments) behind every lexical normalizer: resolve_rel(base, rel, floor) fails closed; normalize_rel_keeping_escapes keeps ../ so a containment check can name the escape; normalize_lexically / normalize_lexically_keeping_escapes do the same for Paths; is_anchored is the common absolute/drive check. Each format keeps only its own admission rule (gradle ~/:, sbt any .., cargo socket-owned prefix, and so on).
  2. Home resolution (B66).
    • utils::fs::home_dir() -> Option<PathBuf> returns HOME, then USERPROFILE, when set and rooted, else None; callers probe nothing without a home. home_from_env applies the same rule to an injected environment.
    • Routed through it: coursier/ivy process_home, the go crawler, the ruby crawler's ambient_home, update::channel, the Maven ~/.m2 default (JvmEnv::m2_repo is now Option; every caller skips the Maven repo when there is none), and the Python crawler's pdm/poetry/expand_home seams. pipenv_home_dir keeps Python's own Windows precedence but now also rejects a relative value.
  3. utils::repo_root. Holds search_dirs, ancestor_search_dirs, find_git_repo and GitRepo::config_path (follows a gitdir: file and a worktree's commondir), plus the owner-trust rule.
    • socket.yml lookup is a thin warning wrapper over it.
    • VEX product detection uses it. When the nearest checkout is owned by another user, or the walk stopped below a repository at the home directory, it now adds a warning to DetectResult::warnings instead of silently falling back to the manifest.
    • JVM not_build_root uses ancestor_search_dirs, which keeps walking past a checkout at the project itself (a submodule can still be a module of the build above it) but now stops at home and at the ceilings.
    • The vex --product help text now describes the nearest-checkout rule.
  4. FIFO-safe reads (B74). get reads pnpm-lock.yaml with read_regular_to_string_sync. cargo_workspace::read_manifest and sidecars::maven::read_sidecar keep their lstat check (symlinks still not followed), then read through the non-blocking regular-file helpers.

Behavior changes to note

  • Home stop precedence is unchanged from main. The repository walk's home stop reads HOME on Unix and USERPROFILE on Windows (git's rule, as the old policy code did), not the crawler resolver's HOME-then-USERPROFILE order. It now ignores a relative value.
  • Crawler home precedence. home_dir keeps main's order (HOME, then USERPROFILE, on every platform). New: a relative or literal ~ value is skipped instead of used.
  • JVM build-root walk. It climbs physical parents (project_root canonicalized, as git does), so a relative root like . now finds the reactor/Gradle/sbt build above it (the old lexical ancestors() saw none). For a symlinked project directory, the parents are those of the link target, not of the link. The refusal names the ancestor in the caller's own spelling when that spelling reaches it; otherwise it shows the canonical path without Windows' \\?\ prefix.
  • The JVM build-root search no longer reads ~/settings.gradle, ~/pom.xml or ~/build.sbt when the project sits directly under $HOME (git's home rule).

Duplicate copies deleted

Concept Before After
./.. folding loops 12 (fs, npm_crawler private, npm_crawler yarnrc, cargo_workspace, gradle/mod, sbt/build, pypi_requirements, cargo_manifest, cargo_config, composer_crawler, vendor/jvm/gradle, jvm/maven_reactor) 1 (utils::relpath::Segments)
Repo-root ancestor walks 3 (policy, vex/product, maven_repo) 1 (utils::repo_root)
Owner-trust / ceiling / home-stop helpers in policy only moved to utils::repo_root (policy copy deleted)
"HOME then USERPROFILE" crawler home rules in core 9 (fs, go_crawler, ruby ambient_home, coursier process_home, update::channel, maven m2_repo_path_with, python pdm/poetry config, poetry cache/installer, expand_home) 1 rule (utils::fs::home_from_env; home_dir is it over the process env)

Home readers that remain, on purpose, because they follow another tool's own rule: the repository walk's home stop (git), pipenv_home_dir (Python's expanduser), Gradle's passwd-vs-$HOME check (gradle_cache::home_mismatch_with), npm's .npmrc env (patch/redirect/npmrc.rs), the socket-cli config location (utils/socket_cli_config.rs) and update::state's cache-dir chain. The npm crawler's macOS global-prefix fallback (npm_crawler.rs HOME read) is in pm:npm scope (#1008) and is left there; it already skips an empty HOME but would accept a relative one.

Testing

All commands ran in the worktree through /private/tmp/claude-501/heavy-job.sh with CARGO_INCREMENTAL=0, -j4, on macOS:

  • cargo build -p socket-patch-core -p socket-patch-cli --tests: clean.
  • cargo clippy -p socket-patch-core -p socket-patch-cli --all-targets: no lint on any line this PR changes (checked by intersecting every clippy location with the diff against the merge base). The macOS toolchain reports lints that are already on main (for example unix_default unused in pdm_dir_candidates). Linux clippy -D warnings is left to CI.
  • cargo test -p socket-patch-core --lib: 5595 passed, 1 failed. The failure is utils::digest::tests::production_digests_go_through_the_helpers, which already fails on main and is fixed by Fix main CI red on stale digest pending-list entries #1016.
  • cargo test -p socket-patch-core --test crawler_maven_e2e --test crawler_gradle_e2e --test crawler_python_e2e --test crawler_cargo_e2e --test telemetry_helpers_e2e: all passed. The first round also ran crawler_{nuget,ruby,go,composer}_e2e and covgap_crawlers_composer_crawler: all passed.
  • cargo test -p socket-patch-cli --lib: 862 passed.
  • cargo test -p socket-patch-cli --test e2e_socket_yml_policy --test covgap_commands_vex --test e2e_vex --test e2e_embedded_vex --test vex_terminal_output --test vendor_jvm_cli --test maven_sidecar_cli --test covgap_commands_get --test in_process_get_modes --test gradle_agent_cli --test contract_gradle_codes --test e2e_scan --test in_process_scan: all passed (11 ignored in e2e_scan). The first round also ran cli_parse_scan, cli_parse_vex, policy_pypi_names, scan_pnpm_relocated_store_cwd_e2e, in_process_cargo_apply and scan_requirements_lock_only: all passed.
  • Failing-first and new tests:
    • detect_in_submodule_names_the_submodule and detect_in_linked_worktree_uses_the_common_config were run against the old product.rs. Both failed: the submodule got pkg:github/acme/superproject and the worktree got None.
    • detect_below_a_home_repository_warns_and_uses_the_manifest: a dotfiles repo at HOME above the project gives the manifest PURL plus a warning; run from HOME itself, the repo is the product.
    • home_dir_never_returns_a_relative_path (old code returned "~"), m2_repo_path_with_no_usable_home_names_no_repository (old code returned ~/.m2/repository) and injected_home_must_be_rooted (python seams; old code returned Library/… / .config/pypoetry under the CWD for HOME="").
    • not_build_root_walks_a_relative_root_and_names_the_callers_spelling: absolute and relative project roots under a parent Gradle build give the refusal with the caller's spelling of the root.
    • keeping_escapes_keeps_every_leading_parent covers the old ../../x → x collapse.
    • FIFO guards: filter_to_installed_purls_pnpm_pnp_hosted_fifo_lock_does_not_wedge (watchdog thread that fails the test instead of hanging), read_manifest_rejects_a_fifo_without_blocking, read_sidecar_rejects_a_fifo_without_blocking. These are guards, not failing-first: a FIFO present from the start is already rejected by the lock inventory or the lstat on main. The B74 change closes the window where a FIFO is swapped in after those checks, which no test races.
    • utils::relpath and utils::repo_root unit tests: fold, floor, keep-escapes, anchored spellings; nearest .git dir or file, home stop (including the reported skipped home repo), ceilings, the ancestor walk past a checkout at the start, submodule/worktree config resolution, and the owner rule.

The intermediate commits were not each built on their own; the final tree was. Please squash-merge.

Deferred

🤖 Generated with Claude Code


Note

Medium Risk
Touches home/Maven local repo resolution and many crawler path probes across ecosystems; behavior changes are intentional (no usable home, relative roots, git product detection) but could skip caches or shift warnings in edge CI/container environments.

Overview
Centralizes duplicated path, home, and git-checkout logic so crawlers, policy, VEX, and JVM vendor checks behave consistently and fail closed on unsafe paths.

utils::relpath replaces a dozen hand-rolled ./.. normalizers (composer, npm/pnpm, gradle, sbt, cargo, PyPI includes, etc.) with one stack-based implementation, including a fix where pnpm’s old normalizer collapsed ../../x to x. utils::fs::home_dir now returns Option<PathBuf> and only accepts rooted HOME/USERPROFILE, so callers no longer probe CWD-relative ~ or ./~/.m2 (B66); JvmEnv::m2_repo is optional and wired through Maven/Gradle apply and scan paths.

utils::repo_root is the single git ancestor walk (ceilings, home stop, owner trust, gitdir:/worktree config). Policy socket.yml lookup, VEX --product git detection, and JVM not_build_root all use it—submodules and worktrees name themselves, dotfiles at $HOME don’t claim every project (with warnings), and build-root search walks physical parents so relative . still finds the reactor above.

FIFO-safe reads (B74): hosted get reads pnpm-lock.yaml via read_regular_to_string_sync; Cargo manifest and Maven sidecar reads use the same regular-file helpers after lstat, with new unix tests so a FIFO can’t wedge the suite.

Reviewed by Cursor Bugbot for commit e13f87a. Configure here.


Generated by Claude Code

Twelve hand-rolled `.`/`..` folding loops (fs::normalize_lexically, the
pnpm crawler's private copy, cargo_workspace::normalize_rel,
gradle::resolve_rel, sbt normalize_dir, pypi normalize_rel_path,
cargo_manifest::normalize_socket_path, cargo_config::path_is_socket_owned,
composer normalize_config_vendor_dir, the yarnrc modules-folder resolver,
vendor/jvm/gradle resolve_dir and maven_reactor normalize) each had its own
rules. They now share one segment stack in utils::relpath: resolve_rel
(fail closed past a floor), normalize_rel_keeping_escapes (keep the
escape so a containment check can name it), normalize_lexically and
normalize_lexically_keeping_escapes for Paths. Each format keeps only its
own admission check (what spellings it refuses up front).

Behavior fix: the pnpm crawler's copy let a second leading `..` pop the
first (PathBuf::pop removes a `..` segment), so `../../x` collapsed to
`x`; the shared keep-escapes normalizer keeps every leading parent.

Audit: architecture audit §3.B (path normalizers).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
utils::fs::home_dir fell back to the literal relative path `~` when
HOME and USERPROFILE were unset, so every `home_dir().join(...)` probe
(cargo registry and config chain, nuget, deno, ruby, python user
site-packages, pipx/uv/pdm/conda/pyenv roots) resolved against the
process cwd: `./~/.cargo/config.toml` inside the scanned project was read
as the user's cargo config. It now returns Option<PathBuf> holding only
an absolute HOME/USERPROFILE, and every caller probes nothing without one.

The coursier/ivy `process_home` wrapper (the one caller that already
filtered for absolute), the go crawler's and the ruby crawler's private
HOME/USERPROFILE readers and update::channel's copy now route through the
same resolver.

Audit: B66 (fs.rs home_dir relative fallback), C19 (home resolvers).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
get's pnpm PnP hosted narrowing read pnpm-lock.yaml with a plain
read_to_string, so a FIFO at that path wedged `get` in open(2); the
hosted flow it claims to match already reads it through
read_regular_to_string_sync. The cargo workspace manifest reader and the
maven checksum-sidecar reader checked `lstat` and then opened with a
blocking read, a hand-rolled copy of the regular-file guard with a
check-then-open race; they keep the lstat (symlinks are still not
followed) but now read through the non-blocking regular-file helpers.

Audit: B74.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Three ancestor walks decided "the repository this directory belongs to"
with different rules. policy::find_repo_root_with_warnings honored
`.git` files, the home-directory stop, GIT_CEILING_DIRECTORIES and owner
trust; the VEX product detector only looked for `<dir>/.git/config` and
walked to `/`; the JVM not_build_root check stopped at any `.git` but
crossed the home directory and the ceilings.

utils::repo_root now owns the walk (search_dirs / ancestor_search_dirs /
find_git_repo) and the owner-trust rule, and resolves a checkout's git
config through a `gitdir:` file and a worktree's `commondir`. All three
callers route through it; the copies in policy and vex/product.rs are
deleted. Behavior fixes for VEX product detection (B22):

- a submodule names its own origin instead of the superproject's;
- a linked worktree reads the shared repository's remotes instead of
  falling back to a manifest (or nothing);
- a dotfiles repository at $HOME is no longer every project's product,
  and GIT_CEILING_DIRECTORIES and an untrusted owner are honored;
- the config is read with the FIFO-safe reader.

The JVM build-root search keeps walking past a checkout at the project
itself (a submodule can be a module of the build above it) but now stops
at the home directory and the ceilings like every other lookup.

Audit: B22, §3.B (repo-root walks).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mikolalysenko Mikola Lysenko (mikolalysenko) added the arch-refactor PR opened by the scheduled architecture refactor routine label Oct 7, 2026
m2_repo_path_with fell back to the literal `~` (and accepted a relative
HOME), so with no usable home the Maven local repository resolved to the
CWD-relative ./~/.m2/repository inside the scanned project (B66). It now
returns None and JvmEnv::m2_repo is optional; every caller skips the
Maven repo when there is none.

The Python crawler's injected-env HOME readers (pdm/poetry config,
cache and installer dirs, expand_home, pipenv's home) took an empty or
relative HOME as-is. They now share utils::fs::home_from_env, the same
rooted-and-non-empty rule as home_dir.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
VEX product detection now adds a warning when the nearest checkout is
owned by another user, or when the walk stopped below a repository at
the home directory, instead of silently falling back to the manifest.

The repository walk's home stop reads HOME on Unix and USERPROFILE on
Windows again (the old policy rule, which is git's), rather than the
crawler home resolver's HOME-then-USERPROFILE order, so an MSYS/Cygwin
HOME does not move the socket.yml stop.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
not_build_root walks canonical parents, so its refusal printed
/private/var/... on macOS and \\?\C:\... on Windows. It now shows the
caller's own lexical ancestor when that names the same directory, else
the canonical path without the verbatim prefix. A test covers a relative
project root, which the old lexical ancestors() walk could not climb.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Also drop the doc comment orphaned by the normalizer test removal in
pypi_requirements.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit e13f87a. Configure here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arch-refactor PR opened by the scheduled architecture refactor routine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants