[codex] migrate retrieval manifest to lexical schema

[TheGreenCedar]

Context

PR #996 replaced the removed lexical sidecar with project-local SQLite FTS, but retrieval_index_manifest still physically stored and updated the producer version through the legacy column. Several response-only serde aliases and an unconditional legacy Compose interpolation also kept removed vocabulary active.

Closes #1043
Refs #983
Refs #899

What Changed

• Added schema 21 after Persist parser source snapshots and reject changed-during-parse results #978's schema-20 file-content migration and renamed the manifest column in place to lexical_version.
• Switched manifest SELECT and UPSERT paths to the canonical column; fresh databases now create only that column.
• Preserved schema-19, schema-20, and interrupted-incremental rows while keeping the sentinel fence intact.
• Removed non-persistent legacy serde aliases and synthetic-hit handling.
• Limited the removed Compose interpolation to owned legacy compose files that actually reference it; current files emit no legacy variable.
• Kept the v0.15 state alias and bounded, ownership-checked artifact cleanup as migration-only input, with v0.16 removal documented.
• Updated the changelog and v0.15 architecture/testing documentation.

How To Review

1. Review the schema order in storage_impl/schema.rs: v20 adds file.content_hash; v21 performs the idempotent manifest rename.
2. Check that retrieval_manifest.rs reads and writes only lexical_version.
3. Review the v18/v19/v20 and interrupted-fence tests for row preservation and repeat writes.
4. Confirm current sidecar serialization/Compose state emits lexical names while legacy cleanup remains ownership checked.

Verification

• cargo fmt --all -- --check
• cargo test -p codestory-store — 86 passed
• cargo test -p codestory-retrieval compose_down_ — 2 passed
• cargo test -p codestory-retrieval legacy_lexical — 2 passed
• cargo test -p codestory-retrieval — 341 unit tests passed, 2 explicitly ignored; integration suites passed 2 + 2 + 4, with 1 live-sidecar test explicitly ignored
• cargo clippy -p codestory-store -p codestory-retrieval --all-targets -- -D warnings
• node .github/scripts/check-doc-links.mjs
• node .github/scripts/check-workflow-policy.mjs
• python -m py_compile .github/scripts/check-packaged-agent-proof.py
• git diff --check

Risk

• Schema 21 intentionally makes older binaries fail closed instead of reopening a database whose column contract they do not understand; the rename preserves every row.
• Legacy state/Compose reads remain only for the documented v0.15 migration window and should be deleted in v0.16.
• Live sidecar integration was not exercised locally; this change was covered by full retrieval tests and CI owns cross-platform publication/runtime checks.