[codex] Harden merge reliability gates and semantic coverage

[adamziel]

Summary

This PR hardens ForkPress merge reliability across the nine problem areas we identified: WordPress semantic objects, plugin semantics, review-only schema cases, filesystem behavior, crash consistency, branch birth metadata, ID bands, stale audits, and release gates.

The main product stance is conservative: merge should apply exact source changes, preserve target state, or produce an auditable conflict. It should not silently rewrite WordPress IDs or plugin data graphs to make conflicts disappear.

What Changed

• Expanded real WordPress semantic coverage for posts/pages, postmeta, comments, users, taxonomy terms/relationships, menus, reusable blocks/synced patterns, page-to-reusable-block references, options, media uploads, generated attachment files, widgets, CPT references, and branch UI create/merge flows.
• Added plugin merge validator infrastructure and coverage for plugin-owned graphs spanning custom tables, JSON, serialized PHP payloads, postmeta/options, referenced CPT data, and files.
• Hardened ID-band behavior so branch-local AUTOINCREMENT IDs are separated before references are written, out-of-band explicit IDs are review-held, and child/reference rows plus inserted or updated scalar/serialized/theme/widget option, post, postmeta, comment, commentmeta, usermeta, termmeta, term-taxonomy, term-relationship, post-author, reusable/media/avatar/navigation/query block-content, taxonomy menu-item, and comment-user references are held with their unsafe owner IDs.
• Required branch birth metadata before merge/reuse, made Git-created branch publication capture ID bands, DB merge bases, file merge bases, and row identities before the branch becomes visible, and made normal branch-create retries discard stale unpublished birth artifacts and reset markers before allocating fresh metadata. Public E2E now exits normal branch creation after birth metadata capture and verifies retry publishes fresh DB/file bases and ID bands.
• Added crash recovery coverage for DB, metadata, filesystem, whole-branch DB+file state, Git-created branch publication, Git branch update/delete cleanup, Git object pruning, public forkpress branch merge kill/recover/retry paths for DB, metadata, before-file, and filesystem checkpoints, public interrupted recovery cleanup retry coverage, and public branch reset crash/retry coverage with pending-reset guards that block merge/reuse after interrupted reset publication.
• Improved stale-audit flow with revalidate-reviews, merge-audit --revalidate, guarded merge-resolve --after-revalidate, plugin validator evidence revalidation, plugin replacement conflict links, and conservative revalidation_class metadata for stale reviewed conflicts, including target drift, source drift, missing target rows/files, incompatible no-primary-key rowid replacements, and supported WordPress same-key semantic replacements.
• Hardened filesystem merge behavior for unsafe paths, symlinks, directory/file and file/directory replacements, upload metadata drift, missing media files, generated upload files, and stale filesystem review revalidation.
• Added release preflight checks so dist builds fail early when static-PHP prerequisites are missing, covered macOS/Linux release prerequisite parsing without PHP headers, and fixed the aarch64 macOS dist build path to avoid empty bash-array expansion under set -u.
• Added docs that spell out the merge model, repair policy, stale audit workflow, crash consistency boundary, plugin validator contract, and remaining reliability gaps.
• Added an objective-to-artifact audit in docs/merge-reliability.md so each original merge reliability requirement maps to concrete tests/docs and an explicit remaining gap.

Merging Model

ForkPress treats merge as a three-surface operation:

• SQLite rows/cells/schema objects
• branch filesystem entries
• external merge metadata and recovery artifacts

Branch IDs are not repaired after the fact. Each branch gets a separate AUTOINCREMENT band before writes happen because WordPress and plugins store IDs in JSON, serialized PHP, block markup, options, and custom tables. If a source row tries to bring an explicit ID outside its reserved band, ForkPress holds that row and dependent references for review instead of rewriting embedded IDs.

Plugin data is not guessed by the generic SQLite layer. The generic merge can preserve rows and files, but plugin validators are the boundary for plugin-specific coherence. A validator can say the staged candidate is coherent, report review conflicts with plugin evidence, or fail the merge and trigger rollback.

Reviewed conflict resolution is guarded. If the target no longer matches the audited payload, resolution stops. Revalidation can carry prior reviewer intent back into needs-action, but drift does not auto-apply. Generic DB row/cell and filesystem conflicts now record compatible-source-drift when the source side changes after review, so a stale source choice is reopened instead of applying old reviewer intent to new source data. Supported WordPress primary-key row conflicts also record incompatible when the same key now describes a different semantic object. Source- and target-side regression coverage now spans every currently implemented WordPress row fingerprint: posts, options, postmeta, terms, term taxonomy, termmeta, users, usermeta, comments, and commentmeta.

Upsides

• Safer default behavior: ambiguous semantic changes become auditable instead of lossy.
• Much broader regression coverage for real WordPress object graphs and plugin-shaped data.
• Separate ID bands avoid the dangerous class of collisions that cannot be repaired reliably after IDs are embedded.
• Git-created and UI-created branches now follow the same birth invariants as CLI branches.
• Crash recovery is explicit: pending recovery artifacts block later mutation until inspected/restored.
• Interrupted branch reset publication is explicit: a pending-reset marker blocks public merge/reuse instead of letting stale merge bases drive a later merge.

Trade-offs

• Some merges produce more review conflicts than a naive row merger would.
• Plugin correctness depends on validators or plugin-owned merge drivers for plugin-specific graphs.
• Schema dependency cases remain conservative when automatic ordering would be ambiguous.
• Release confidence now depends on platform gates, especially APFS/aarch64 macOS behavior, not just Linux PHP tests.

Still Missing

• Real-plugin validators for known cross-table, serialized, JSON, and file graphs; plugin-owned merge drivers only where deterministic repair can be proven.
• Broader external kill harnesses for actual Git push/serve entry points, beyond the representative smart-HTTP Git-created branch-list crash now covered in E2E, with recovery verified from a fresh process after interruption.
• Broader explicit-ID/import enforcement beyond covered AUTOINCREMENT row insert/rewrite and known WordPress reference cases.
• Richer plugin/schema stale-audit evidence, plugin/custom semantic identity classifiers, and guarded plugin/schema resolution flows where appropriate.
• More deterministic schema dependency planning for safe view/trigger reorderings while keeping cyclic or semantic ambiguity review-only.
• APFS sparsebundle detach/compact kill coverage and a green post-fix aarch64 macOS release artifact gate. Follow-up work is tracked in Follow-up: finish merge reliability hardening after PR #46 #48: Follow-up: finish merge reliability hardening after PR #46 #48.

Validation

• umask 022; php tests/cow/merge.php passed with 2912 assertions.
• php -l scripts/cow/merge.php
• php -l tests/cow/merge.php
• bash -n scripts/build-dist.sh
• make test-release
• make test-release PHP_CONFIG= PHP_DEV_DIR= PKG_CONFIG=
• make test-branchfs PHP_CONFIG= PHP_DEV_DIR= PKG_CONFIG= fails with the expected PHP-header error.
• bash -n tests/release/build-dist-preflight.sh
• umask 022; CARGO_TARGET_DIR=/tmp/forkpress-target-merge-reliability cargo test -p forkpress-storage
• bash -n tests/cow/e2e.sh
• umask 022; php tests/cow/branch_ui.php passed with 25 assertions.
• php -l scripts/cow/git_server.php
• php -l tests/cow/git_server.php
• umask 022; php tests/cow/git_server.php passed with 252 assertions.
• umask 022; CARGO_TARGET_DIR=/tmp/forkpress-cli-test.<tmp> FORKPRESS_RUNTIME_BUNDLE=/dev/null cargo test -p forkpress-cli --bin forkpress passed with 61 tests.
• git diff --check

[adamziel]

Follow-up tracker for the remaining merge reliability work: #48

[adamziel]

Wrapping PR #46 here as the merge-reliability hardening milestone.

Current state:

• Branch is pushed at 9f72db434c02c468706509cb692d0f4c0bbe988f.
• Worktree is clean and aligned with origin/codex/merge-reliability-hardening.
• PR is open, not draft, and mergeable.
• Local focused validation is captured in the PR body, including the 2907-assertion PHP merge suite, Git server suite, CLI unit tests, release preflight checks, and syntax/diff checks.
• CI is still running on the latest push; no failed check was visible at wrap time.

Follow-up work is documented in #48 and linked from the PR body. That issue is intentionally about what remains before claiming fully reliable real-world merging, especially real-plugin validators, broader product-level crash harnesses, APFS sparsebundle kill coverage, explicit-ID/import expansion, richer plugin/schema stale-audit evidence, deterministic schema dependency planning, and macOS release gates.

[adamziel]

Rebased PR #46 onto current origin/trunk for the Linux storage changes and force-pushed the replayed branch.

New head: a37f87ded0701aa3a783a8b89ce9ac356c0f0112.

Conflict resolution notes:

• Kept trunk’s new Linux/XFS storage imports and tests.
• Kept this branch’s merge crash recovery, stale revalidation, and plugin validator CLI imports.
• Took trunk’s deletion of the old docs/storage-drivers.md page, since storage docs now live under docs/storage/.

Post-rebase validation:

• grep -R -n '<<<<<<<\\|=======\\|>>>>>>>' ... found no conflict markers in touched runtime/test/doc files.
• git diff --check origin/trunk...HEAD
• php -l scripts/cow/merge.php
• bash -n tests/cow/e2e.sh
• umask 022; CARGO_TARGET_DIR=/tmp/forkpress-storage-rebase-test.<tmp> cargo test -p forkpress-storage passed: 16 tests.
• umask 022; CARGO_TARGET_DIR=/tmp/forkpress-cli-rebase-test.<tmp> FORKPRESS_RUNTIME_BUNDLE=/dev/null cargo test -p forkpress-cli --bin forkpress passed: 61 tests.
• umask 022; php tests/cow/git_server.php passed: 252 assertions.
• umask 022; php tests/cow/merge.php passed: 2907 assertions.

CI restarted on the rebased head; no failed check was visible immediately after the push.

[adamziel]

Pushed 0d841b6 after the rebase CI failure. The failing WP UI E2E path was using the nonce from the main admin page without preserving the auto-login cookies, so admin-post verified against a different WordPress session and returned the expired-link 403. The test now keeps a curl cookie jar across the nonce fetch and branch create/merge submit, matching browser behavior.\n\nAdditional local validation after this patch:\n- bash -n tests/cow/e2e.sh\n- git diff --check\n- php -l tests/cow/merge.php\n- umask 022; php tests/cow/branch_ui.php passed with 25 assertions\n- umask 022; php tests/cow/merge.php passed with 2912 assertions\n\nCI restarted on the new head and is currently in progress.