Tracking: classify line endings in one place instead of five drifting rules

[mikolalysenko]

[agent] Filed by the scheduled architecture audit routine (ecosystems and formats). Register: register comment.

Kind: tracking. Source: review 4.4 ("CRLF policy is inconsistent"), 5.4 (toml_edit CRLF), 7.3; register E16.

Problem

Every writer that splices into a text file has to answer one question: "which line terminator does this file use?" On main @ 045d7ec the crate answers it with five different rules, plus about a dozen inline copies:

Rule	Where	Mixed CRLF/LF file
Any \r\n → CRLF	common::detect_eol (go.sum, go.mod, requirements, yarn classic); a byte-identical private copy, pypi_uv::newline_of; and inline copies in formats/gem/hosted.rs#L65, maven_reactor.rs#L1519, [`#L1776`](https://github.com/SocketDev/socket-patch/blob/045d7ec783d788bf3c5a1310724b51e09fb6505d/crates/socket-patch-core/src/vendor/jvm/maven_reactor.rs#L1776),`` upstream/composer.rs#L259, [`upstream/gem.rs#L535`](https://github.com/SocketDev/socket-patch/blob/045d7ec783d788bf3c5a1310724b51e09fb6505d/crates/socket-patch-core/src/patch/redirect/upstream/gem.rs#L535),`` upstream/pypi.rs#L629, [`upstream/cargo.rs#L62`](https://github.com/SocketDev/socket-patch/blob/045d7ec783d788bf3c5a1310724b51e09fb6505d/crates/socket-patch-core/src/patch/redirect/upstream/cargo.rs#L62-L63)`` and #L304,`` redirect/pipenv.rs#L129, `redirect/npmrc.rs#L614`, `redirect/mod.rs#L5775`, `utils/python_script.rs#L44`	CRLF
First line's terminator	gradle::newline_of; yarn classic block_eol (first line of the block)	whatever the first line uses
Majority	LineEndings + majority_terminator, used by JsonLayout (common.rs#L217-L222) and composer lock_text.rs#L35-L40``	majority (tie → LF)
Re-expand only CRLF-only input	python_lock::preserve_line_endings (15 toml_edit callers in utils, redirect and vendor); hosted cargo crlf_to_lf	whole rendering LF (toml_edit) / refused (hosted cargo)
Refuse	vendored pnpm refuses any CRLF (pnpm_lock.rs#L524); berry refuses mixed	refused

The rules have drifted. A throwaway unit probe on main, run twice, gives three different answers for the same input:

"a\nb\r\nc\r\n": detect_eol → CRLF, pypi_uv::newline_of → CRLF, gradle::newline_of → LF, majority_terminator → CRLF, preserve_line_endings → every line LF
"a\r\nb\nc\n":   detect_eol → CRLF, pypi_uv::newline_of → CRLF, gradle::newline_of → CRLF, majority_terminator → LF,   preserve_line_endings → every line LF

They also disagree within one ecosystem. For Cargo, the hosted rewrite refuses a mixed Cargo.lock (cargo_mixed_line_endings_still_refuse), but upstream restore LF-normalizes it and re-expands every line to CRLF (upstream/cargo.rs#L62-L63, [`#L155`](https://github.com/SocketDev/socket-patch/blob/045d7ec783d788bf3c5a1310724b51e09fb6505d/crates/socket-patch-core/src/patch/redirect/upstream/cargo.rs#L155)).`` For yarn classic, hosted CRLF-expands the whole file while vendored splices with detect_eol and reverts with block_eol (#467).

Symptoms

• Yarn classic hosted and vendored rewrites convert LF lines of a mixed CRLF/LF yarn.lock to CRLF, so rollback is not byte-exact #467: a mixed yarn classic lock is rewritten wholesale (hosted) or loses byte-exact revert (vendored).
• Poetry and PDM lock rewrites flip a mixed-line-ending lock's edited unit in opposite directions #695: Poetry and PDM flip a mixed lock's edited unit in opposite directions. PR Fix Poetry/PDM lock splice drift (#694, #695) #703 adds a sixth rule (the majority of the replaced fragment).
• Vendored pnpm rewrites a CRLF package.json as LF, and vendor --revert does not restore it #662: vendored pnpm package.json loses CRLF (serialize_json), in PR Fix vendored pnpm package.json CRLF/BOM layout (#662) #810.

Impact

Uniform files are safe: every rule agrees on an LF-only or CRLF-only file, which is what git autocrlf produces. The risk is on mixed files (editor merges on Windows), where each new writer picks a rule by copy-paste. That produces churn and broken byte-exact reverts, one bug report per writer. Size: about 25 sites, mostly one-liners.

Target design

utils::line_endings is the only place that classifies terminators:

• LineEndings::of (exists);
• fn terminator(text) -> &'static str: Crlf → \r\n, Mixed → majority_terminator, otherwise \n. It's stable under appending lines in its own style, so a revert that removes {line}{nl} still finds what the forward pass wrote;
• fn restore_rendering(original, rendered): the toml_edit re-expansion, moved from python_lock, with one documented mixed-file rule (per replaced fragment, as Fix Poetry/PDM lock splice drift (#694, #695) #703 does).

Whether a writer refuses a mixed file stays a per-format decision, but it reads the same classifier.

Checklist

• Child 1: spell the insert terminator once (line_endings::terminator), deleting common::detect_eol, both newline_of copies and the inline copies. Pick the line terminator for spliced lines through one line_endings::terminator #815
• Child 2: move preserve_line_endings and hosted cargo's crlf_to_lf into utils::line_endings (restore_rendering / LineEndings::of), adopting the per-fragment rule from Fix Poetry/PDM lock splice drift (#694, #695) #703 for toml_edit output. After Fix Poetry/PDM lock splice drift (#694, #695) #703.
• Child 3: yarn classic hosted and vendored use the block-local terminator (block_eol) on both forward and revert (fixes Yarn classic hosted and vendored rewrites convert LF lines of a mixed CRLF/LF yarn.lock to CRLF, so rollback is not byte-exact #467). After E08 (yarn grammars).
• Child 4: Cargo upstream restore uses the hosted classifier, so a mixed lock is refused or kept, not CRLF-expanded.
• Vendored pnpm's CRLF refusal goes with formats/pnpm/lines.rs (review 4.7 G). It's tracked under that refactor, not here.

Dependencies

Child 1 can start now. Child 2 is blocked by PR #703. Child 3 should follow PR #657 and E08.

[mikolalysenko]

[agent] Triaged: priority:p3 (a cross-cutting refactor). This is a tracking issue. Its children (#815 so far) and the related bug #467 are worked as separate slices. Not a duplicate, and no open PR covers it.

---

Generated by Claude Code

[mikolalysenko]

[agent] Poetry bug-hunt routine (ledger #311): here's a concrete user-visible symptom of the preserve_line_endings row for child 2. It's still present on main 99f61d2, after #703.

PR #703 fixed the forward splice: on a mixed lock, scan --mode hosted now changes only the six unit. Hosted rollback and remove don't go through that engine, though. restore_poetry writes the whole re-rendered document back through preserve_line_endings (upstream/pypi_locks.rs#L177). For a mixed input, that keeps toml_edit's all-LF rendering, so every line of the lock goes to LF, including packages the patch never touched.

Repro (Linux, real Poetry 2.5.1 / lock 2.1 and 1.8.5 / lock 2.0, local mock patch API, SOCKET_PYPI_JSON_API forwarder). The lock locks six, idna and certifi. It's CRLF except the certifi block, which is LF (31 CRLF / 11 LF lines on 2.5.1; 29 / 10 on 1.8.5):

socket-patch scan --mode hosted --yes --ecosystems pypi   # only the six unit changes (good, #703)
socket-patch rollback --yes                               # exit 0
# lock before scan: CRLF 31, LF 11  ->  after rollback: CRLF 0, LF 42 (content equal modulo EOL)
socket-patch remove pkg:pypi/six@1.16.0 --yes             # same: CRLF 29 / LF 10 -> CRLF 0 / LF 39 (1.8.5 lock)

I reproduced it twice per Poetry version, with both rollback and remove. poetry check --lock and poetry install still pass afterwards, so the cost is a whole-file diff on a revert that should be exact, not a broken install. An all-LF or all-CRLF lock rolls back byte for byte, as the tracking issue predicts. The same finish path also serves restore_pdm and the vendored→hosted takeover restore, so I expect PDM to behave the same way, but I only tested Poetry.

I'm not filing this separately, because child 2 here covers it. Please make sure the per-fragment rule reaches upstream/pypi_locks.rs::finish too, not only the forward splice.

---

Generated by Claude Code