Add the parity suite as a scheduled conformance report

.github/workflows/parity.yml

@@ -0,0 +1,99 @@
+name: Parity
+
+# Scheduled conformance report: neokapi's Go ports head-to-head against this
+# bridge, using a released kapi and a plugin built from this checkout. The
+# suite lives under parity/ and compiles against a released neokapi module.
+#
+# This workflow reports; it gates nothing. It has no pull_request trigger, so
+# it never blocks a change here, and nothing in neokapi/neokapi consults it.
+#
+# Security: the dispatch inputs are consumed only through env vars, never
+# inlined into `run:` strings.
+
+on:
+  schedule:
+    # Weekly, Monday 05:00 UTC.
+    - cron: "0 5 * * 1"
+  workflow_dispatch:
+    inputs:
+      okapi_version:
+        description: "Okapi Framework version to build the plugin for"
+        required: false
+        default: "1.48.0"
+      kapi_version:
+        description: "kapi release to test ('latest' = newest stable, or e.g. 1.3.0-rc9)"
+        required: false
+        default: "latest"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: false
+
+permissions:
+  contents: read
+
+jobs:
+  parity:
+    name: parity-test
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    env:
+      OKAPI_VERSION: ${{ inputs.okapi_version || '1.48.0' }}
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Set up Java 17
+        uses: actions/setup-java@v5
+        with:
+          distribution: temurin
+          java-version: "17"
+          cache: maven
+
+      - name: Set up Go
+        uses: actions/setup-go@v6
+        with:
+          go-version-file: parity/go.mod
+          cache-dependency-path: parity/go.sum
+
+      - name: Install a released kapi
+        uses: neokapi/setup-kapi@v1
+        with:
+          version: ${{ inputs.kapi_version || 'latest' }}
+          plugins: ""
+          cache-tm: "false"
+
+      - name: Regenerate the version-specific POM
+        # okapi-releases/<V>/pom.xml references the parent version at the time
+        # it was committed; regenerate so it matches the root POM at HEAD.
+        run: make generate-pom V="$OKAPI_VERSION"
+
+      - name: Install the parent POM into the local Maven repository
+        # The version module declares neokapi:okapi-bridge-parent as its
+        # parent and `mvn package` does not install it first.
+        run: mvn -B install -N -DskipTests -q
+
+      - name: Build the plugin tarball
+        run: |
+          make build V="$OKAPI_VERSION"
+          make plugin-v2 V="$OKAPI_VERSION"
+
+      - name: Run the parity suite
+        run: make parity-test
+
+      - name: Write the per-filter summary
+        if: always()
+        run: |
+          if [ -f .parity/test-comparison.json ]; then
+            cd parity && go run ./cmd/testcompare -in ../.parity/test-comparison.json -out ../.parity/parity-report.json
+          fi
+
+      - name: Upload the parity report
+        if: always()
+        uses: actions/upload-artifact@v6
+        with:
+          name: parity-report
+          path: |
+            .parity/test-comparison.json
+            .parity/parity-report.json
+          if-no-files-found: warn
+          retention-days: 90

.gitignore

@@ -21,3 +21,6 @@ dist/
 
 # Transient fleet-authoring manifest (per-id gap list)
 .gap-manifest.json
+
+# Parity sandbox (make parity-sandbox)
+.parity/

CLAUDE.md

@@ -155,3 +155,25 @@ The parent `pom.xml` declares all infrastructure dependencies (gRPC, Gson, Snake
 - Schema `version` is an integer N that increments on content change; `$id` includes the version
 - CI on push to main: regenerates composites and verifies schemas are up-to-date (fails if out of date)
 - CI on tag push: matrix build across all 11 Okapi versions
+
+## Parity suite (`parity/`)
+
+`parity/` is a separate Go module (`github.com/neokapi/okapi-bridge/parity`)
+that runs neokapi's Go ports head-to-head against this bridge. It requires a
+released `github.com/neokapi/neokapi` and nothing from neokapi's `host` or
+`cli` modules; the daemon launcher is the plugin protocol's Mode C handshake
+implemented locally. Format specs (`core/formats/<id>/spec.yaml`) are read from
+that module's source directory (`go list -m`), so specs and readers come from
+one version. Per-fixture divergence annotations live in `parity/annotations/`.
+
+```bash
+make parity-vet        # compile + vet the suite, no sandbox needed
+make parity-test       # sandbox (released kapi + fresh plugin tarball) + full suite
+make parity-publish    # suite + per-filter summary under .parity/
+```
+
+The suite is behind the `parity` build tag, so `make test-go` never runs it.
+`.github/workflows/parity.yml` runs it weekly and on dispatch, installs kapi
+with `neokapi/setup-kapi@v1`, and uploads the report as an artifact. It is a
+report, not a gate: it has no pull_request trigger, and nothing in the neokapi
+repository consults it. See `parity/README.md`.

Makefile

@@ -60,6 +60,14 @@ help:
 	@echo "Release:"
 	@echo "  make release              Release current version (bump, tag, push, snapshot)"
 	@echo ""
+	@echo "Parity (parity/, neokapi's Go ports against this bridge):"
+	@echo "  make parity-sandbox       Build .parity/ from a released kapi + a fresh plugin tarball"
+	@echo "  make parity-test          Run the parity suite against the sandbox"
+	@echo "  make parity-publish       Suite + per-filter summary (.parity/parity-report.json)"
+	@echo "  make parity-vet           Compile and vet the suite without a sandbox"
+	@echo "  make parity-clean         Remove the sandbox"
+	@echo "  make regen-okapi-fixtures OKAPI_REPO=...  Re-harvest fixtures from the Okapi Java tests"
+	@echo ""
 	@echo "Supported versions: $(SUPPORTED_VERSIONS)"
 	@echo "Latest version: $(LATEST_VERSION)"
 
@@ -417,3 +425,6 @@ release:
 	@echo "Release complete: v$(CURRENT_VERSION)"
 	@echo "  Tag: v$(CURRENT_VERSION)"
 	@echo "  Next dev: $(NEXT)-SNAPSHOT"
+
+# The parity suite (parity/): neokapi's Go ports against this bridge.
+include parity/parity.mk

parity/README.md

@@ -0,0 +1,112 @@
+# Parity suite
+
+This module runs neokapi's Go ports of Okapi filters and steps head-to-head
+against the Okapi Framework, through this repository's bridge plugin and, when
+a build is reachable, the tikal CLI. It answers one question per filter: given
+the same input and the same semantic configuration, do the two implementations
+produce the same content?
+
+The suite is a scheduled conformance report, not a gate. It runs weekly from
+`.github/workflows/parity.yml`, which installs a released kapi, builds the
+plugin from this checkout, runs the suite and uploads the report as an
+artifact. No pull request here or in `neokapi/neokapi` waits on it.
+
+## Layout
+
+| Path | Contents |
+| --- | --- |
+| `*.go` | The harness: sandbox resolution, daemon launch, native and bridge runners, the canonical comparison, the report collector |
+| `spec/` | `ParityRunner`, which drives a format's `spec.yaml` through both implementations |
+| `formats/` | One spec test per filter, the harvested fixtures, and the `bridge_config` translators |
+| `roundtrip/` | The three-engine round-trip harness (see `docs/roundtrip-testing.md`) |
+| `tools/` | The step table: every bridge step run for stability |
+| `annotations/` | Per-format fixture annotations: severity, issue link and skip directives for known divergences |
+| `cmd/testcompare` | Turns the raw report into the per-filter summary |
+| `cmd/okapi-test-scan` | Harvests `@Test` snippets from the Okapi Java sources into `formats/fixtures_*_generated.go` |
+| `pseudobench/` | Pseudo-translation throughput of kapi with and without the bridge over the Okapi corpus |
+| `scripts/sandbox.sh` | Builds the sandbox the suite runs against |
+
+## Dependencies
+
+`go.mod` requires a released `github.com/neokapi/neokapi` and nothing from the
+`host` or `cli` modules: the suite compiles from the module proxy, and the
+daemon launcher is the plugin protocol's Mode C handshake implemented here.
+Format specs (`core/formats/<id>/spec.yaml`) are read from that module's
+source directory, which `NeokapiDir` resolves with `go list -m`, so the specs
+and the readers under test come from one version.
+
+Bump the pin when a new kapi release is out:
+
+```bash
+cd parity && go get github.com/neokapi/neokapi@v1.3.0 && go mod tidy
+```
+
+## Running locally
+
+```bash
+make parity-test                 # sandbox + full suite
+make parity-publish              # suite + per-filter summary under .parity/
+KAPI_BIN=/path/to/kapi make parity-test
+OKAPI_VERSION=1.47.0 make parity-test
+PARITY_FORCE=1 make parity-sandbox   # rebuild the sandbox
+```
+
+The sandbox lives under `.parity/` (gitignored):
+
+```
+.parity/
+├── bin/kapi                  the kapi under test (copied from $KAPI_BIN or PATH)
+├── plugins/okapi-bridge/     the plugin tarball built by `make plugin-v2`, unpacked
+├── okapi-testdata/<version>/ the okapi-testdata release (binary fixtures)
+└── test-comparison.json      the raw report
+```
+
+Tests resolve the sandbox from `KAPI_PARITY_SANDBOX`, or by walking up from
+the working directory to a `.parity/bin/kapi`, and fail when there is none.
+`KAPI_PARITY_SKIP=1` turns that failure into a skip for runs that do not care
+about parity. Nothing in the suite reads the user's installed plugins.
+
+One format:
+
+```bash
+cd parity && KAPI_PARITY_SANDBOX=$PWD/../.parity \
+  go test -tags "fts5,parity" -count=1 -run TestParityHtmlSpec ./formats/
+```
+
+Every tag the suite needs is spelled in one `-tags` flag; `go` keeps only the
+last occurrence of the flag.
+
+## Reading the report
+
+`test-comparison.json` holds one row per filter or step and pass:
+
+- `format`: the native reader against the bridge reader on the same input,
+  compared on the canonical projection (part types, block ids, translatable
+  flag, source and target text with structural placeholders for inline codes).
+- `format-roundtrip`: read then write, compared on bytes, where both sides
+  declare a writer.
+- `format-tikal`: the native round trip against an extract-then-merge run of
+  the Okapi CLI, when `OKAPI_TIKAL` points at a launcher.
+- `format-fixture`: one row per harvested upstream `@Test` fixture.
+- `step`: a bridge step run against a fixed input for stability.
+
+`expected_fail` and `parity_warn` are documented divergences, not regressions;
+the annotation under `annotations/<format>.yaml` or the `expected_fail` entry
+in the format's `spec.yaml` says why. `cmd/testcompare` folds the rows into
+`parity-report.json`: one row per filter or step with totals.
+
+## Adding a parity case
+
+1. Add or extend the format's `spec.yaml` in neokapi (`core/formats/<id>/`):
+   the bridge filter class, the tikal corner and the parity skips are declared
+   there.
+2. Add `formats/<id>_spec_test.go` here, wiring the native reader for a
+   head-to-head comparison or leaving it unset for a bridge-only snapshot. Add
+   `<id>_bridge_config.go` when the bridge's parameter shape differs.
+3. For a step, add a row to the table in `tools/spec.go`.
+4. Run the suite. A pure default-only difference is converged with explicit
+   configuration in the bridge config translator, not recorded as an expected
+   failure.
+
+The harvested fixtures are regenerated with `make regen-okapi-fixtures
+OKAPI_REPO=/path/to/Okapi`; review the diff before committing.

parity/annotations/csv.yaml

@@ -0,0 +1,20 @@
+format: csv
+fixtures:
+  computer_science_article.csv:
+    severity: fixture-bug
+    summary: "Source contains Windows-1252 byte 0x96 (en-dash) with no explicit charset marker. Native preserves the raw byte; okapi treats it as invalid UTF-8 and replaces with U+FFFD. Neither is 'right' for parity without an explicit encoding annotation."
+    skip:
+      engines: [okapi]
+      reason: "fixture has Windows-1252 0x96 with no charset marker; engines disagree on invalid-byte handling"
+  some_blank_columns.csv:
+    severity: cosmetic
+    summary: "Source rows have layout `one,,four,,seven` (empty translatable column). With sourceColumns=2 okapi falls back to writing column-1's translation into the empty column-2 slot; native leaves the empty cell empty. Two defensible behaviours."
+    skip:
+      engines: [okapi]
+      reason: "okapi writes col-1 translation into adjacent empty cell; native leaves empty cells empty"
+  test2cols.csv:
+    severity: fixture-bug
+    summary: "Fixture is intentionally malformed CSV (unmatched double-quotes spanning newlines, embedded `\"\"` inside quoted regions). Native and okapi recover from the broken quote state differently — both are defensible readings of broken input."
+    skip:
+      engines: [okapi]
+      reason: "fixture is malformed CSV (quotes spanning lines); engines recover from broken state differently"

parity/annotations/html.yaml

@@ -0,0 +1,6 @@
+format: html
+fixtures:
+  France_Culture_fr.html:
+    severity: cosmetic
+    summary: "Non-deterministic html-canonical divergence, PRE-EXISTING and unrelated to the openxml faithful-writer work (html reader/writer + html-canonical normalizer untouched). The fixture flips between canonical-equal and divergent across identical-code runs: when divergent, native emits ~837 extra bytes around the document's Open Graph <meta property=\"…\"> tags (e.g. article:publisher) in word/head reconstruction. The difference is document metadata only — it renders identically — so it is cosmetic. The non-determinism (native html-writer head/meta emission ordering) is a separate test-stability issue to investigate; annotated here so the dashboard documents it whichever way the flake lands."
+    spec_ref: "HTML5 §4.2.5 (the meta element); §13 (parsing)"

parity/annotations/json.yaml

@@ -0,0 +1,8 @@
+format: json
+fixtures:
+  invalid_by_most_processors.json:
+    severity: fixture-bug
+    summary: "Intentionally-malformed JSON5 fixture with bare keys and single-quoted strings. Both engines emit lenient output; json-canonical normalizer can't parse either side, so there's no useful tier comparison."
+    skip:
+      engines: [okapi]
+      reason: "JSON5 fixture; both engines emit lenient output, json-canonical can't parse either side"

parity/annotations/odf.yaml

@@ -0,0 +1,20 @@
+format: odf
+# Measured over the 15 upstream okapi/filters/openoffice fixtures:
+#
+#   odf | native | 15 total | 0 byte | 15 canon (15 faithful) | 0 divergent
+#   odf | bridge | 15 total | 15 byte | 0 canon | 0 divergent
+#
+# Native reaches canonical-equal rather than byte-equal because okapi
+# re-serializes every stream on round-trip — reordering attributes, rewriting
+# the declaration, expanding empty elements — while native writes each stream
+# from the source's own bytes. Native is the more faithful side, which is why
+# the format is classified CanonFaithful and why driving these to byte-equal
+# against okapi would be a regression: an untranslated .odt/.ods/.odp/.odg
+# comes back byte for byte against the SOURCE, on all 15
+# (core/formats/odf/package_roundtrip_test.go).
+#
+# Bridge is byte-equal on all 15 (it literally invokes Okapi's
+# OpenOfficeFilter via okapi-bridge#11's two-pass pipeline).
+#
+# No per-fixture divergences.
+fixtures: {}