diff --git a/CHANGELOG.md b/CHANGELOG.md index 34967d15..819daae2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## Unreleased +## v0.4.1 + +Audit findings D01-D16, E01-E04, L01-L06, P01, and G01 against `d7dc98b7ee99fc46b767c93444a9f051fdbea2ba`. Do not retag v0.4.0. Frozen confirmation files and their SHA-256 sums are unchanged. + - Defensive hidden-Unicode scanner and cleaner (`fuckmark-hidden-scan-v1`). `fuckmark --scan` audits any text for hidden or suspicious Unicode without changing it; `fuckmark --clean` strips it while keeping the visible text. Coverage is general, not FuckMark-only: bidirectional controls (Trojan Source, CVE-2021-42574), zero-width and invisible spacing, Unicode tag characters (hidden-text / prompt-injection smuggling), variation selectors, enclosing marks, deprecated interlinear controls, other `Cf` format controls, C0/C1 controls, private-use codepoints, and noncharacters. Tab, newline, carriage return, and space are never flagged, and ordinary combining accents (`Mn`) are left alone. - `--scan` prints a human report by default, a machine line with `-q`, and a `fuckmark-scan ...` status line to stderr with `--status`. `--clean` removes every flagged category (so it reverses a FuckMark mix back to the visible text) and reports the count removed. `--detect`, `--scan`, and `--clean` are mutually exclusive; neither `--scan` nor `--clean` combines with `--visible`. - `fuckmark web` serves the same engine at `POST /api/scan`. New Python API: `scan_hidden_characters`, `clean_hidden_characters`, `classify_hidden_codepoint`, `scan_dict`, `scan_human_report`, `scan_machine_line`, `ScanResult`, and `HiddenFinding` (exported from `fuckmark` and `fuckmark.product`). @@ -22,12 +26,7 @@ - Audit fixes: `fuckmark lint` and `--scan`/`--clean`/`guard`/`normalize` now decode CESU-8 / WTF-8 lone surrogates (`utf-8` with `surrogatepass`) instead of skipping or crashing; those files fail the default `surrogate` category. The GitHub Action passes `select` / `args` / `paths` through environment variables so workflow inputs are not interpolated into the bash script. Clipboard reads keep UTF-8 NULs on Linux/macOS; only PowerShell `Get-Clipboard` uses a BOM-less UTF-16LE heuristic so Windows ASCII is not read as `U+0000` controls. JSON responses and receipts escape lone surrogates as `\uXXXX` so the bytes stay valid UTF-8 (`Content-Type: application/json; charset=utf-8`). `sha256_text` uses `surrogatepass` so a lone surrogate no longer raises `UnicodeEncodeError`. `fuckmark web` requires `Content-Length` on POST bodies. The measurement-calibration builder imports `MeasurementCalibrationCorpus` for its return annotation. - `fuckmark normalize` (`fuckmark-normalize-v1`): NFC, optional identifier lookalike fold (UTS #39-inspired subset, not a full confusable map), then strip of the security category set, plus a JSON receipt (`input_sha256` / `output_sha256` / `steps` / `report_hash`). `--clean` remains the blunt strip with no receipt. HTTP: `POST /api/normalize`. Python: `normalize_text`, `skeleton_fold`. Reference: `docs/normalize.md`. - VS Code / Cursor: `fuckmark.cleanOnSave` (default off) strips hidden Unicode on save. -- Frozen confirmation files, hashes, mix mechanisms, and the published wheel are unchanged. Package version stays 0.4.1. - -## v0.4.1 - -Audit findings D01-D16, E01-E04, L01-L06, P01, and G01 against `d7dc98b7ee99fc46b767c93444a9f051fdbea2ba`. Do not retag v0.4.0. Frozen confirmation files and their SHA-256 sums are unchanged. - +- Frozen confirmation files, hashes, mix mechanisms, and the published v0.4.0 wheel are unchanged. Package version stays 0.4.1. - Live mix is five-layer (`u034f-ufe00-cc-me-cf-ia-letter-alt-v1`, `release-cli-v12`): each eligible grapheme cluster gets U+034F or U+FE00, a C0/C1 control, enclosing Me (U+20DD), a cycling Egyptian hieroglyph format control (U+13430-U+13438), and a cycling interlinear annotation control (U+FFF9-U+FFFB). UnicodeSanitizer turns those annotation controls into spaces, so Mn then Me then UnicodeSanitizer then frozen Cf-strip no longer restores the source. Sites are Latin, Greek, Cyrillic, Han, Kana, Hangul syllables, and emoji. NFD Latin is marked after the combining sequence so normalization does not split the cluster. Visible projection stays identical. Mn-strip, default-ignorable strip, UnicodeSanitizer orderings, Mn then Me then UnicodeSanitizer, and the required sanitizer bundle no longer restore the source. Chromium pre-pixel probes reject the live Me-layer mix; historical mark-only remains pixel-equal where measured. The Egyptian Cf cycle stays the Unicode 14 assigned range U+13430-U+13438. - Exploratory restore census of frozen Gate v2 watermarked sources on seeds 1200000, 1210000, and 1220000 (n=192, detector not run): four-layer **0/192** restore under Mn then Me then UnicodeSanitizer, and **0/192** match to `UnicodeSanitizer(source)`, with Cf residual on every row. Historical triple-layer under that same path matches `UnicodeSanitizer(source)` **192/192**. Confirmation artifacts were not rewritten. - Exploratory GPT-2 / SynthID rescore of those same frozen sources remains historical triple-layer evidence (`u034f-ufe00-cc-me-letter-alt-v1`): **0/192** on raw, Mn-strip, DI-strip, UnicodeSanitizer, Mn then US, DI then US, US then Mn, required-bundle, and bundle then US. Historical dual-layer / mark-only under Mn then US returns to **182/192** detections. @@ -38,7 +37,7 @@ Audit findings D01-D16, E01-E04, L01-L06, P01, and G01 against `d7dc98b7ee99fc46 - `fuckmark web` serves that local browser tool for beginners who prefer a page over the CLI (`http://127.0.0.1:8765/mark.html`). Detect and strip go through a local Python API (`GET /api/health`, `POST /api/remove-marks`) that calls `detect_fuckmark_insertions` and `project_visible_v1`. Static `mark.q1z.org` has no Python process and keeps the in-browser fallback. - Product authorization v2 records the five-layer path. Gate v2 confirmation hashes stay historical. The v1 mix publishability report records the historical triple-layer fixture measurements and remains `product_authorized: false`. - Public copy, installer messages, and this source tree describe carrier insertion. They no longer say the CLI returns text unchanged. `docs/website.md` is the controlled website/installer copy. Do not pipe `https://d.q1z.org/mark` into a shell. -- Package version is 0.4.1. The published v0.4.0 wheel remains the last GitHub Release artifact and does not implement `--text` / `--file`. +- Package version is 0.4.1. Do not retag v0.4.0. The v0.4.0 wheel remains published and does not implement `--text` / `--file`. - Markdown reference labels may span lines and line endings (LF, CRLF, CR). Definitions may put the destination on the next line and may appear in blockquotes or lists. Multiline inline destinations are protected. HTML tags, HTML entities, and indented code are protected. - Extensionless relative paths such as `scripts/build`, last-component spaces such as `C:/My final notes.txt` and `C:/Users/Alice/My final notes.txt`, and `ftp://` URIs are protected. `and/or` remains eligible. - CLI outcomes: stderr always reports processed vs not processed, reason, insertions, sites, `last_index`, `source_length`, and capped unless `-q`, including stream success. Successful transforms note that Mn-strip, default-ignorable strip, UnicodeSanitizer combinations, and Cf-strip after UnicodeSanitizer leave Me/Cc/Cf residuals and spaces. `--status` includes `processed`, `source_length`, and `first_unsupported`, including for `too-large` and internal failure. `--inspect` prints a character-level map. `--detect` scans for approved insertions without transforming. Clipboard partial success is exit 3. Usage errors remain exit 2. Internal transform failure is exit 4. Output is written before clipboard copy. Stdout write failures no longer escape as a traceback. diff --git a/docs/release.md b/docs/release.md index c3dc42bc..efd7b247 100644 --- a/docs/release.md +++ b/docs/release.md @@ -19,7 +19,7 @@ FuckMark v0.4.1 is the audit-fix and five-layer mix release on top of the Gate v 1. Merge this v0.4.1 branch to green `main`. Clone install (`python -m pip install .`) already runs the product CLI from that tree. The GitHub Release wheel URL is valid only after the next two steps. 2. On the resulting `main` push, rerun the cross-platform package matrix. That push must **not** create tags, publish a GitHub Release, or delete branches. 3. Create and push the immutable `v0.4.1` tag on that merge commit yourself. The workflow never runs `git tag`. Do not tag a pull-request SHA if `main` will be a squash merge. Do not retag `v0.4.0`. -4. Publish with `workflow_dispatch` and `publish_github_release=true` on that same commit. The job refuses to run unless `v0.4.1` already exists and points at the dispatch SHA. If the GitHub Release is missing, it uploads the verified wheel and sdist. It does not delete merged branches. +4. Publish with `workflow_dispatch` and `publish_github_release=true` on that same commit. The job refuses to run unless `v0.4.1` already exists and points at the dispatch SHA. If the GitHub Release is missing, it uploads the verified wheel, sdist, and `SHA256SUMS.txt` written by `tools/verify_release_install.py` into `dist/`. It does not delete merged branches. 5. Record the published v0.4.1 wheel SHA-256 in README and `docs/install.md` in a follow-up commit, as was done for v0.4.0. 6. Deploy [mark.q1z.org](https://mark.q1z.org), `https://mark.q1z.org/demo.html` from `docs/demo.html`, and `https://d.q1z.org/mark` from `docs/website.md`, including the PowerShell User-Agent response. The demo must stay static, work from `file://`, and must not run detectors on visitor paste. diff --git a/tests/test_install_release_hardening.py b/tests/test_install_release_hardening.py index 916d6fbe..28111e25 100644 --- a/tests/test_install_release_hardening.py +++ b/tests/test_install_release_hardening.py @@ -55,6 +55,13 @@ def test_release_engineering_does_not_auto_tag_publish_or_delete_branches() -> N assert "workflow_dispatch" in text assert "github.event_name == 'workflow_dispatch'" in text assert "inputs.publish_github_release == true" in text + assert "python tools/verify_release_install.py dist" in text + assert "name: release-distributions" in text + assert "path: dist/" in text + assert 'gh release create "$RELEASE_TAG" release-dist/*' in text + verify = VERIFY_RELEASE_INSTALL.read_text(encoding="utf-8") + assert "_write_checksums(directory, artifacts)" in verify + assert "SHA256SUMS.txt" in verify def test_workflow_dispatch_inputs_are_not_interpolated_into_run_scripts() -> None: