Add pdf form, docx and xlsx handler + tests

CLAUDE.md

@@ -38,7 +38,8 @@ adapters/               # concrete port impls — the only place frameworks live
   validator_jsonschema.py #  Draft 2020-12 gate
   nats_transport.py     #   publish results / DLQ
   render_executor.py    #   Inline (tests) + ProcessPool (prod)
-  renderers/            #   jinja_html_pdf (WeasyPrint), xlsx (openpyxl), registry, _paths (traversal guard)
+  renderers/            #   jinja_html_pdf (WeasyPrint), xlsx (openpyxl), docx (docxtpl),
+                        #   pdf_form (pypdf), registry, _paths (traversal guard), _ooxml (zip repack)
 core/                   # infra cloned/trimmed from simulation-key
   config.py  queue/{init.py}  storage/client.py  logging.py  tracing.py  metrics.py
 worker/
@@ -67,10 +68,26 @@ of truth for both `error.permanent` and the ack/nak decision.
   so `./logo.png`/`./invoice.css`/fonts resolve. Templates receive `data` +
   `locale`; `StrictUndefined` makes a missing field a hard error (no silent blanks).
 - `xlsx` → openpyxl: workbook **defined names** matching top-level `data` keys,
-  plus `data["cells"]` (`"A1"` / `"Sheet!A1"` → value, applied last).
-- Manifest has two **optional** fields beyond the spec: `entrypoint` (default
-  `template.html` / `template.xlsx`) and `output_basename` (default = last
-  dot-segment of `id`, e.g. `billing.invoice` → `invoice.pdf`).
+  manifest-declared repeating **blocks**, then `data["cells"]` (`"A1"` /
+  `"Sheet!A1"` → value, applied last).
+- `docx` → docxtpl (Jinja inside a real `.docx`). Pure Python — unlike
+  WeasyPrint it imports and tests fine on a bare host. `autoescape=True` is
+  mandatory: a `.docx` body is XML, so an unescaped `&` corrupts the package.
+- `pdf-form` → pypdf: fills the authority's own AcroForm. `manifest.fields` maps
+  a data key to the (auto-generated) field name; checkbox export states are read
+  from the PDF, not configured.
+- Manifest optional fields beyond the spec: `entrypoint` (default
+  `template.html` / `.xlsx` / `.docx` / `.pdf`), `output_basename` (default =
+  last dot-segment of `id`), `blocks` (xlsx), `fields` (pdf-form).
+- **Blocks write in place, never `insert_rows`** — openpyxl does not fix up
+  formulas, merged ranges, defined names, data validations or print areas on a
+  shift, and regulator forms are full of all of them. Capacity is
+  `required_fields.properties.<source>.maxItems`; overflow is a **permanent**
+  `VALIDATION_ERROR` (a retry of the same list can never fit). Unused rows are
+  hidden, not deleted.
+- **Adding an engine is guarded at both ends**: `domain/models.py` raises at
+  import if `_DEFAULT_ENTRYPOINT` misses an `Engine`, and
+  `tests/renderers/test_registry.py` asserts every `Engine` has a renderer.
 
 ## Reliability (JetStream)
 

README.md

@@ -47,7 +47,8 @@ adapters/      # concrete implementations of the ports
   validator_jsonschema.py  Draft 2020-12 validation gate
   nats_transport.py        publish results / DLQ
   render_executor.py       inline + process-pool execution
-  renderers/               jinja-html→PDF (WeasyPrint), xlsx (openpyxl), registry
+  renderers/               jinja-html→PDF (WeasyPrint), xlsx (openpyxl),
+                           docx (docxtpl), pdf-form (pypdf), registry, _ooxml
 core/          # reused infra (settings, logging, tracing, metrics, queue, storage client)
 worker/        # main.py entrypoint + dispatcher (subscription, ack/nak/DLQ)
 ```
@@ -106,20 +107,59 @@ Error codes: `VALIDATION_ERROR`, `TEMPLATE_NOT_FOUND`, `UNSUPPORTED_FORMAT`
 }
 ```
 
-- `engine` — `jinja-html` (PDF/HTML via Jinja2 + WeasyPrint) or `xlsx` (openpyxl).
-- `entrypoint` *(optional)* — entry file; defaults `template.html` / `template.xlsx`.
+- `engine` — one of:
+
+  | engine | produces | library | use for |
+  |---|---|---|---|
+  | `jinja-html` | `pdf`, `html` | Jinja2 + WeasyPrint | documents we author |
+  | `xlsx` | `xlsx` | openpyxl | spreadsheet forms |
+  | `docx` | `docx` | docxtpl | documents whose mandated format is Word |
+  | `pdf-form` | `pdf` | pypdf | filling an authority's own fillable PDF |
+
+- `entrypoint` *(optional)* — entry file; defaults `template.html` /
+  `template.xlsx` / `template.docx` / `template.pdf`.
 - `output_basename` *(optional)* — artifact base name; defaults to the last
   dot-segment of `id` (`billing.invoice` → `invoice.pdf`).
 - `required_fields` — JSON Schema; an empty schema validates everything.
+- `blocks` *(xlsx only)* — repeating regions, see below.
+- `fields` *(pdf-form only)* — data key → AcroForm field name.
 
 **jinja-html templates** receive the request `data` as `data` and the locale as
 `locale` (e.g. `{{ data.invoice_number }}`). Relative assets (`./invoice.css`,
 `./logo.png`, fonts) resolve against the template directory. Missing fields are a
 hard error (no silent blanks). See `tests/fixtures/templates/billing_invoice/`.
 
-**xlsx templates** are filled from `data` two ways: workbook **defined names**
-matching top-level `data` keys, and an explicit `data.cells` map of
-`"A1"` / `"Sheet!A1"` → value (applied last).
+**docx templates** work the same way — `{{ data.x }}` written inside Word.
+
+**xlsx templates** are filled from `data` three ways: workbook **defined names**
+matching top-level `data` keys, **repeating blocks**, and an explicit
+`data.cells` map of `"A1"` / `"Sheet!A1"` → value (applied last).
+
+A block turns a variable-length list into worksheet rows. Layout is declared by
+the *manifest*, never by the caller — the anchor cell is a fact about the
+workbook:
+
+```json
+"blocks": [{
+  "source": "participants",
+  "anchor": "Membres et actionnaires!A4",
+  "columns": ["categorie", "nom", "rue", "code_postal"],
+  "hide_unused_rows": true
+}]
+```
+
+`data.participants` is then a list of dicts and row *i* column *j* is
+`rows[i][columns[j]]`. Rows are written **in place** into a band the template
+already provisions — never inserted, because openpyxl does not fix up formulas,
+merged ranges, defined names, data validations or print areas when rows shift.
+Capacity comes from `required_fields.properties.<source>.maxItems` (or an
+explicit `max_rows`), and overflowing it is a **permanent** `VALIDATION_ERROR`
+raised before any render. Unused rows are hidden, not deleted.
+
+**pdf-form templates** are the authority's own fillable PDF; only field values
+change. Official forms auto-name their fields, so the manifest maps them:
+`"fields": {"community_name": "Champ de texte 68"}`. Checkbox export states are
+read from the PDF itself, so `/Oui`, `/Yes` and `/1` all work without config.
 
 ## Reliability (JetStream)
 

adapters/renderers/_ooxml.py

@@ -0,0 +1,51 @@
+"""Deterministic repacking for OOXML (Office Open XML) documents.
+
+``.xlsx`` and ``.docx`` are both OPC packages: a zip whose members include
+``docProps/core.xml`` carrying ``<dcterms:created>``/``<dcterms:modified>``.
+Both writers we use stamp wall-clock time — openpyxl rewrites ``core.xml``'s
+modified date inside ``save()``, and both libraries let ``zipfile`` stamp each
+member with ``time.localtime()``. Neither is a property of (template, data), so
+neither may reach the bytes we hash.
+
+Shared by the xlsx and docx renderers so the two cannot drift.
+"""
+
+from __future__ import annotations
+
+import io
+import re
+import zipfile
+
+# Fixed (not wall-clock) so normalised packages are reproducible.
+_FIXED_ZIP_DATE = (1980, 1, 1, 0, 0, 0)
+_FIXED_ISO = b"2000-01-01T00:00:00Z"
+_CREATED_RE = re.compile(rb"(<dcterms:created[^>]*>)[^<]*(</dcterms:created>)")
+_MODIFIED_RE = re.compile(rb"(<dcterms:modified[^>]*>)[^<]*(</dcterms:modified>)")
+
+_CORE_PROPS = "docProps/core.xml"
+# OPC requires the content-types stream to be the first part in the package.
+# Sorted order happens to put it first today ('[' is 0x5B, below every other
+# part's initial letter), but relying on that is a latent corruption bug — so
+# emit it explicitly first instead.
+_CONTENT_TYPES = "[Content_Types].xml"
+
+
+def normalise_ooxml_bytes(raw: bytes) -> bytes:
+    """Repack an OOXML package so identical input yields identical bytes."""
+    with zipfile.ZipFile(io.BytesIO(raw)) as src:
+        members = {name: src.read(name) for name in src.namelist()}
+
+    ordered = [name for name in (_CONTENT_TYPES,) if name in members]
+    ordered += sorted(name for name in members if name != _CONTENT_TYPES)
+
+    out = io.BytesIO()
+    with zipfile.ZipFile(out, "w", zipfile.ZIP_DEFLATED) as dst:
+        for name in ordered:
+            data = members[name]
+            if name == _CORE_PROPS:
+                data = _CREATED_RE.sub(rb"\g<1>" + _FIXED_ISO + rb"\g<2>", data)
+                data = _MODIFIED_RE.sub(rb"\g<1>" + _FIXED_ISO + rb"\g<2>", data)
+            info = zipfile.ZipInfo(name, date_time=_FIXED_ZIP_DATE)
+            info.compress_type = zipfile.ZIP_DEFLATED
+            dst.writestr(info, data)
+    return out.getvalue()

adapters/renderers/docx.py

@@ -0,0 +1,102 @@
+"""Renderer for the ``docx`` engine: Jinja inside a real ``.docx`` via docxtpl.
+
+For documents whose mandated format is Word — the CWaPE standard DSO agreements,
+for instance — where a PDF would not be accepted. Authoring matches the
+``jinja-html`` bundles: the same ``{{ data.x }}`` expressions, but written inside
+Word rather than in HTML.
+
+``StrictUndefined`` mirrors the jinja-html renderer: a missing field is a hard
+error, never a silent blank, which is the right default for an official document.
+
+Determinism: docxtpl introduces nothing wall-clock of its own, but python-docx
+lets ``zipfile`` stamp members with the current time, so the package is repacked
+through ``_ooxml.normalise_ooxml_bytes`` — the same pass the xlsx renderer uses,
+since both are OPC packages.
+
+Unlike WeasyPrint, docxtpl is pure Python (its only compiled dependency is lxml,
+whose wheels bundle libxml2/libxslt), so this renderer imports and its tests run
+on a bare developer host. It is still imported lazily, matching the convention.
+"""
+
+from __future__ import annotations
+
+import datetime
+import io
+from collections.abc import Mapping
+from typing import Any
+
+from adapters.renderers._ooxml import normalise_ooxml_bytes
+from adapters.renderers._paths import resolve_template_file
+from domain.errors import DocGenError, RenderError
+from domain.models import OutputFormat, RenderedArtifact
+from domain.ports import TemplateBundle
+
+_FIXED_TIMESTAMP = datetime.datetime(2000, 1, 1)
+_NORMALISED_AUTHOR = "document-generation"
+
+
+class DocxRenderer:
+    """Stateless (hence picklable for the process pool) docx renderer."""
+
+    def render(
+        self,
+        bundle: TemplateBundle,
+        data: Mapping[str, Any],
+        fmt: OutputFormat,
+        *,
+        locale: str | None,
+    ) -> RenderedArtifact:
+        if fmt is not OutputFormat.DOCX:
+            raise RenderError(f"docx engine cannot produce {fmt.value!r}")
+        try:
+            content = self._render(bundle, dict(data), locale)
+        except DocGenError:
+            raise
+        except Exception as exc:  # any docxtpl/python-docx failure → transient
+            raise RenderError(f"docx render failed: {exc}") from exc
+
+        return RenderedArtifact(
+            format=fmt,
+            filename=bundle.manifest.filename_for(fmt),
+            content=content,
+        )
+
+    @classmethod
+    def _render(cls, bundle: TemplateBundle, data: dict[str, Any], locale: str | None) -> bytes:
+        # Lazy import, and the Jinja Environment is built here rather than held
+        # on the instance: an Environment is not reliably picklable and this
+        # renderer crosses the process-pool boundary.
+        from docxtpl import DocxTemplate  # type: ignore[import-untyped]
+        from jinja2 import Environment, StrictUndefined
+
+        entry = resolve_template_file(bundle.root, bundle.manifest.resolve_entrypoint())
+        template = DocxTemplate(str(entry))
+        # autoescape is mandatory here, not a lint appeasement: a .docx body is
+        # XML, so an unescaped "&" or "<" in a value (think "Smith & Co")
+        # produces a corrupt package Word refuses to open.
+        #
+        # finalize maps None to an empty string. Jinja's default is to print the
+        # literal "None", which in a contract reads as a filled-in value and is
+        # far worse than a visible blank. StrictUndefined still catches a field
+        # the caller forgot entirely — the two guard different mistakes.
+        environment = Environment(
+            undefined=StrictUndefined,
+            autoescape=True,
+            finalize=lambda value: "" if value is None else value,
+        )
+        template.render({"data": data, "locale": locale}, environment)
+        cls._normalise_properties(template.docx)
+
+        buffer = io.BytesIO()
+        template.save(buffer)
+        return normalise_ooxml_bytes(buffer.getvalue())
+
+    @staticmethod
+    def _normalise_properties(document: Any) -> None:
+        """Clear author/revision metadata so the template author does not leak."""
+        props = document.core_properties
+        props.created = _FIXED_TIMESTAMP
+        props.modified = _FIXED_TIMESTAMP
+        props.last_modified_by = _NORMALISED_AUTHOR
+        props.author = _NORMALISED_AUTHOR
+        props.revision = 1