From 86c14c95cd8555916f01de0d9ef1f9d827b2f9da Mon Sep 17 00:00:00 2001 From: Odinn Adalsteinsson Date: Sun, 25 Oct 2026 18:36:21 +0100 Subject: [PATCH 1/4] =?UTF-8?q?fix:=20overs=C3=A6t=20reply-links=20ved=20k?= =?UTF-8?q?onvertering?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Konverterede beskeder får nye numre, så kopierede reply-links kunne pege på andre beskeder eller på sig selv. Opret først beskederne uden nummerlinks, og oversæt derefter links med writernes faktiske målidentiteter. Fjern links til udeladte beskeder, og returnér revisioner efter opdateringen. --- CHANGELOG.md | 4 + README.md | 4 + docs/JSON.md | 24 ++- docs/SPEC.md | 16 +- scripts/build_reading_pages.py | 23 ++- src/golded_ftn_tools/api.py | 54 ++++++- src/golded_ftn_tools/reply_links.py | 79 ++++++++++ tests/test_api.py | 234 ++++++++++++++++++++++++++++ write.html | 12 +- 9 files changed, 432 insertions(+), 18 deletions(-) create mode 100644 src/golded_ftn_tools/reply_links.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 56b2d63..eb3e3e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## Unreleased + +- Remap numeric reply links when importing export envelopes, including forward links and existing destinations. Drop links to messages outside the batch and return revisions after link updates. + ## 1.1.0 — 2026-10-25 - Add SQLite message-base support and an optional network extra for packet, nodelist and node commands. Operational failures use closed codes, including legacy commands; broken stdout exits 141. Base operations remain available without network dependencies. diff --git a/README.md b/README.md index 6bab9e8..e67b17e 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,10 @@ validation finishes before opening a writer. Each append commits separately; a later failure keeps earlier appends. A broken pipe can hide the receipt of an already committed message. Rerunning may create duplicates. +Export envelopes remap numeric reply links within the input batch and drop +links to omitted messages. Their receipts follow link updates. Flat write +objects keep destination-local reply numbers unchanged. + `read --body` emits the reader's `body_text` exactly, including any controls present there, without adding a newline. `read --revision` adds the writer's identity and revision. `export --archive` reports reader issues as JSONL on diff --git a/docs/JSON.md b/docs/JSON.md index 428828f..1df59dc 100644 --- a/docs/JSON.md +++ b/docs/JSON.md @@ -60,7 +60,8 @@ base number. Do not conceal contradictory controls, MSGID or charset declaration Reject unknown top-level/control keys, duplicate keys, NaN/Infinity, incorrect nested types, BOM and invalid UTF-8 with a record number. `provenance`, `msgno`, `identity`, `revision` and other non-write fields inside an export envelope are -dropped when unwrapping. Flat write objects must not include those fields. +dropped when unwrapping. Source identity and area metadata are used to remap +reply links before being discarded. Flat write objects must not include those fields. Provenance comes from the source/read operation, not an imported claim about the new record. @@ -70,8 +71,25 @@ format. The CLI maps nested `control_lines.kludges` to `control_lines` and neste stripped to match the write date grammar. No arbitrary binary header import or requested message number. Append assigns -the format's number. Fixture authors manage forward reply links; there is no -automatic link resolver. +the format's number. + +For export envelopes, `write` translates `reply_to_msgno`, `reply1st_msgno`, +`reply_next_msgno` and `reply_list` into the destination numbering. Links resolve +only within the same source format, base, board and area, and only to messages +in the same input batch. Missing targets are dropped. Forward links work, as do +imports into an existing base. Duplicate source identities or missing source +format/base/message number are rejected during preflight. + +Conversion appends messages without numeric links, then updates the links using +the actual write results. Conversion receipts follow link updates and contain +the resulting revisions. If an append fails, the committed prefix is linked +using only that prefix before its receipts and the error are emitted. A link +update failure can leave committed messages without all their links; the error +reports the input record and total committed append count. Conversion is not a +batch transaction. Interrupted conversion can leave incomplete links. + +Flat write objects retain destination-local reply numbers unchanged. Fixture +authors manage their forward links. MSGID/REPLY control lines are unchanged. ## Export and read diff --git a/docs/SPEC.md b/docs/SPEC.md index f145df1..051a685 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -86,9 +86,11 @@ reject `--area` for other formats and reject `--board` with sqlite. V1 does not expose Hudson's separate `scan_path`; use the writer package's documented default. Investigate external scan directories before adding an option. -Export envelopes and flat write objects are both accepted. Envelope unwrapping -drops provenance, source msgno, area metadata, identity and revision; the -destination writer assigns new numbers. +Export envelopes and flat write objects are both accepted. The destination +writer assigns new numbers. Envelope source identities and area metadata scope +reply-link remapping within the input batch; they are not imported as target +metadata. Missing link targets are dropped. Flat objects retain target-local +reply numbers. See JSON.md for the source identity and conversion contract. Read, parse and structurally validate the entire input before opening a writer session. Spool normalized, validated JSONL records to a temporary file instead @@ -106,6 +108,14 @@ receipts only for committed messages. stderr gives the failed record number and the committed count. No automatic retry or rollback of earlier successes. No guarantee against process death, power loss or an interrupted pipe. +For batches containing export envelopes, append without the envelopes' numeric +links first, then update those links using actual destination identities. +Emit each receipt after its link update with the resulting revision. On append +failure, finalize links within the committed prefix before reporting the error. +An update failure reports the failing record and committed append count; it +does not roll back the batch. Keep message payloads in the spool; retain only +source identities and write results in memory for link resolution. + A broken pipe can occur after append commits but before its receipt reaches the consumer. Help and docs must explain this: rerunning can create duplicates. diff --git a/scripts/build_reading_pages.py b/scripts/build_reading_pages.py index 5696164..db5e06c 100644 --- a/scripts/build_reading_pages.py +++ b/scripts/build_reading_pages.py @@ -322,7 +322,8 @@ def write_page() -> str: "

Reads one JSON object from stdin and appends it. " "--jsonl reads one JSON value per line and appends each " "value as its own message. Structural checks finish before the writer " - "opens. Each successful append commits on its own and prints a receipt.

" + "opens. Each successful append commits on its own. Export-envelope " + "receipts follow reply-link updates.

" + code_pane( "write-one", "append one message", @@ -359,10 +360,15 @@ def write_page() -> str: "routing_seen_by, routing_patharray of stringsempty. Order and duplicates are kept." "" "

Unknown keys, duplicate keys, NaN, Infinity, a BOM and invalid UTF-8 " - "are rejected. Booleans are not integers. msgno, " - "identity, revision and export envelopes are " - "not write input. Append assigns the message number. There is no " - "automatic reply-link resolver.

" + "are rejected. Booleans are not integers. Flat write objects cannot " + "include msgno, identity or revision. " + "Append assigns the message number. Flat objects keep their " + "destination-local reply numbers.

" + "

Export envelopes are also valid input. Their numeric reply links " + "are remapped within the same source base, board and area in the " + "input batch, including forward links. Links to omitted messages " + "are dropped. Source format, base and message number are required; " + "duplicate source identities are rejected before writing.

" + code_pane("write-message", "examples/message.json", message) + "

posted_at is YYYY-MM-DDTHH:MM:SS with an " "optional Z or numeric UTC offset, and no microseconds. " @@ -396,7 +402,12 @@ def write_page() -> str: "

If record 3 fails after record 1 and 2 committed, those two " "messages stay. Stderr names the failing record and the committed " "count. A broken pipe can hide a receipt for a message that is " - "already stored. Running the same batch again can append duplicates.

", + "already stored. Running the same batch again can append duplicates.

" + "

Conversion first appends messages without numeric links, then " + "updates links using the assigned numbers. Receipts contain the " + "revisions after those updates. An interrupted conversion or failed " + "link update can leave committed messages with incomplete links. " + "The error reports the failing record and committed append count.

", ), ] return shell( diff --git a/src/golded_ftn_tools/api.py b/src/golded_ftn_tools/api.py index f4e1d75..c0d3f25 100644 --- a/src/golded_ftn_tools/api.py +++ b/src/golded_ftn_tools/api.py @@ -14,6 +14,7 @@ ParsedMessage, ReaderIssue, ReaderOptions, + WriteResult, WriterOptions, detect_charset, repair_mojibake, @@ -22,6 +23,7 @@ from . import adapters from .errors import ToolError, classify from .json_contract import InputError, dump, load, outgoing +from .reply_links import SourceMessage, link_patch, source_message, without_links class ExportResult: @@ -130,18 +132,26 @@ def write( encoding: str = "CP850", lock_timeout: float = 5.0, ) -> Iterator[dict[str, object]]: - """Append preflighted records. Yield one write_result per commit.""" + """Append preflighted records and remap reply links in export envelopes.""" _board(format, board, required=True) _area(format, area) resolved = base.resolve() committed = 0 record = 1 + sources: list[SourceMessage | None] = [] + seen: set[SourceMessage] = set() with tempfile.TemporaryFile(mode="w+t", encoding="utf-8", newline="\n") as spool: count = 0 for record, raw in enumerate(records, 1): try: data = load(raw.decode("utf-8", errors="strict")) outgoing(data, format) + source = source_message(data) + if source is not None: + if source in seen: + raise InputError("duplicate source message identity") + seen.add(source) + sources.append(source) spool.write(dump(data)) except InputError as error: raise ToolError( @@ -182,9 +192,29 @@ def write( ), area=area, ) as session: + results: list[WriteResult] = [] + numbers: dict[SourceMessage, int] = {} + failure: ToolError | None = None for record, text in enumerate(spool, 1): - result = session.append(outgoing(load(text), format)) + message = outgoing(load(text), format) + source = sources[record - 1] + try: + result = session.append( + without_links(message) if source is not None else message + ) + except BrokenPipeError: + raise + except Exception as error: + failure = classify( + error, input_record=record, committed=committed + ) + break committed += 1 + if seen: + results.append(result) + if source is not None: + numbers[source] = result.identity.msgno + continue yield { "schema_version": 1, "type": "write_result", @@ -192,6 +222,26 @@ def write( "identity": result.identity, "revision": result.revision, } + if seen: + spool.seek(0) + for record, result in enumerate(results, 1): + message = outgoing(load(spool.readline()), format) + source = sources[record - 1] + if source is not None and without_links(message) != message: + result = session.update( + result.identity, + link_patch(message, source, numbers), + result.revision, + ) + yield { + "schema_version": 1, + "type": "write_result", + "input_record": record, + "identity": result.identity, + "revision": result.revision, + } + if failure is not None: + raise failure except BrokenPipeError: raise except Exception as error: diff --git a/src/golded_ftn_tools/reply_links.py b/src/golded_ftn_tools/reply_links.py new file mode 100644 index 0000000..f50bc74 --- /dev/null +++ b/src/golded_ftn_tools/reply_links.py @@ -0,0 +1,79 @@ +"""Translate area-local reply numbers in exported messages.""" + +from dataclasses import dataclass, replace + +from golded_ftn import UNSET, MessagePatch, OutgoingMessage + +from .json_contract import InputError + + +@dataclass(frozen=True) +class SourceMessage: + format: str + base: str + board: int | None + area_code: str | None + area_meta_key: str | None + msgno: int + + +def source_message(data: dict[str, object]) -> SourceMessage | None: + if data.get("type") != "message": + return None + source, message = data.get("source"), data.get("message") + if not isinstance(source, dict) or not isinstance(message, dict): + raise InputError("export requires source and message objects") + format, base, board = source.get("format"), source.get("base"), source.get("board") + msgno = message.get("msgno") + area_code, area_meta_key = message.get("area_code"), message.get("area_meta_key") + if ( + not isinstance(format, str) + or not format + or not isinstance(base, str) + or not base + or (board is not None and type(board) is not int) + or type(msgno) is not int + or msgno < 1 + or (area_code is not None and not isinstance(area_code, str)) + or (area_meta_key is not None and not isinstance(area_meta_key, str)) + ): + raise InputError("export requires an unambiguous source message identity") + return SourceMessage(format, base, board, area_code, area_meta_key, msgno) + + +def without_links(message: OutgoingMessage) -> OutgoingMessage: + return replace( + message, + reply_to_msgno=None, + reply1st_msgno=None, + reply_next_msgno=None, + reply_list=(), + ) + + +def link_patch( + message: OutgoingMessage, + source: SourceMessage, + numbers: dict[SourceMessage, int], +) -> MessagePatch: + def target(number: int | None) -> int | None: + return numbers.get(replace(source, msgno=number)) if number else None + + return MessagePatch( + reply_to_msgno=target(message.reply_to_msgno) + if message.reply_to_msgno + else UNSET, + reply1st_msgno=target(message.reply1st_msgno) + if message.reply1st_msgno + else UNSET, + reply_next_msgno=target(message.reply_next_msgno) + if message.reply_next_msgno + else UNSET, + reply_list=tuple( + mapped + for number in message.reply_list + if (mapped := target(number)) is not None + ) + if message.reply_list + else UNSET, + ) diff --git a/tests/test_api.py b/tests/test_api.py index a758c53..ee9e2f2 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -2,8 +2,10 @@ import json from pathlib import Path +from typing import cast import pytest +from golded_ftn import MessageIdentity, ParsedMessage from test_cli import message from golded_ftn_tools import catalog, create, export, heads, read, write @@ -11,6 +13,238 @@ from golded_ftn_tools.json_contract import InputError, dump, outgoing +@pytest.mark.parametrize("format", ["sqlite", "msg", "opus", "jam", "squish", "hudson"]) +def test_converted_reply_links_follow_messages(tmp_path: Path, format: str) -> None: + source = tmp_path / "source" + create("msg", source) + list( + write( + "msg", + source, + [ + message(subject="deleted"), + message(subject="parent"), + message(subject="child", reply_to_msgno=2), + ], + ) + ) + (source / "1.msg").unlink() + records = [dump(row).encode() for row in export("msg", source)] + target = tmp_path / "target" + create(format, target, **({"area_code": "TEST"} if format == "sqlite" else {})) + board = 1 if format == "hudson" else None + # Existing messages must not become accidental link targets. + list( + write( + format, + target, + [message(subject="existing"), message(subject="existing too")], + board=board, + ) + ) + receipts = list(write(format, target, records, board=board)) + rows = { + cast(ParsedMessage, row["message"]).subject: cast(ParsedMessage, row["message"]) + for row in export(format, target, board=board) + } + parent, child = rows["parent"], rows["child"] + assert child.reply_to_msgno == parent.msgno + for receipt in receipts: + current = read( + format, + target, + cast(MessageIdentity, receipt["identity"]).msgno, + board=board, + revision=True, + ) + assert isinstance(current, dict) + assert current["revision"] == receipt["revision"] + + +@pytest.mark.parametrize("format", ["sqlite", "msg", "opus", "jam", "squish", "hudson"]) +def test_conversion_resolves_forward_links_and_drops_missing_targets( + tmp_path: Path, format: str +) -> None: + source = tmp_path / "source" + create("msg", source) + list( + write( + "msg", + source, + [ + message(subject="deleted"), + message(subject="parent", reply_to_msgno=1, reply1st_msgno=3), + message(subject="child", reply_to_msgno=2), + ], + ) + ) + (source / "1.msg").unlink() + target = tmp_path / "target" + create(format, target, **({"area_code": "TEST"} if format == "sqlite" else {})) + board = 1 if format == "hudson" else None + list( + write( + format, + target, + [dump(row).encode() for row in export("msg", source)], + board=board, + ) + ) + messages = [ + cast(ParsedMessage, row["message"]) + for row in export(format, target, board=board) + ] + parent, child = messages + assert not parent.reply_to_msgno + assert parent.reply1st_msgno == child.msgno + assert child.reply_to_msgno == parent.msgno + + +def test_conversion_maps_next_reply_and_reply_list(tmp_path: Path) -> None: + target = tmp_path / "target" + create("sqlite", target, area_code="TEST") + records = [] + inputs: list[tuple[int, dict[str, object]]] = [ + (20, {"reply_next_msgno": 40, "reply_list": [99, 40]}), + (40, {"reply_next_msgno": 99}), + ] + for number, fields in inputs: + records.append( + json.dumps( + { + "type": "message", + "source": {"format": "sqlite", "base": "source"}, + "message": {**json.loads(message()), "msgno": number, **fields}, + } + ).encode() + ) + list(write("sqlite", target, records)) + first = read("sqlite", target, 1) + second = read("sqlite", target, 2) + assert isinstance(first, dict) and isinstance(second, dict) + assert cast(ParsedMessage, first["message"]).reply_next_msgno == 2 + assert cast(ParsedMessage, second["message"]).reply_next_msgno is None + # Squish exposes the first reply-list entry as reply1st_msgno. + squish = tmp_path / "squish" + create("squish", squish) + data = json.loads(records[0]) + data["message"].pop("reply_next_msgno") + other = json.loads(records[1]) + other["message"].pop("reply_next_msgno") + list(write("squish", squish, [json.dumps(row).encode() for row in (data, other)])) + first_squish = next(iter(export("squish", squish))) + assert cast(ParsedMessage, first_squish["message"]).reply1st_msgno == 3 + + +@pytest.mark.parametrize("scope", ["base", "board", "area_code", "area_meta_key"]) +def test_conversion_does_not_link_across_source_areas( + tmp_path: Path, scope: str +) -> None: + target = tmp_path / "target" + create("sqlite", target, area_code="TEST") + parent = { + "type": "message", + "source": {"format": "sqlite", "base": "source", "board": 1}, + "message": { + **json.loads(message()), + "msgno": 20, + "area_code": "A", + "area_meta_key": "A", + }, + } + child = json.loads(json.dumps(parent)) + child["message"].update(msgno=40, reply_to_msgno=20) + if scope in {"base", "board"}: + child["source"][scope] = 2 if scope == "board" else "other" + else: + child["message"][scope] = "B" + list(write("sqlite", target, [json.dumps(row).encode() for row in (parent, child)])) + result = read("sqlite", target, 2) + assert isinstance(result, dict) + assert cast(ParsedMessage, result["message"]).reply_to_msgno is None + + +def test_failed_conversion_finalizes_committed_prefix(tmp_path: Path) -> None: + source = tmp_path / "source" + create("sqlite", source, area_code="TEST") + list( + write( + "sqlite", + source, + [message(subject="parent", reply1st_msgno=2), message(subject="x" * 80)], + ) + ) + target = tmp_path / "target" + create("msg", target) + receipts = [] + with pytest.raises(ToolError) as caught: + for receipt in write( + "msg", target, [dump(row).encode() for row in export("sqlite", source)] + ): + receipts.append(receipt) + assert caught.value.input_record == 2 + assert caught.value.committed == 1 + assert len(receipts) == 1 + result = read("msg", target, 1, revision=True) + assert isinstance(result, dict) + assert not cast(ParsedMessage, result["message"]).reply1st_msgno + assert result["revision"] == receipts[0]["revision"] + + +@pytest.mark.parametrize( + "invalid", ["duplicate", "missing-source", "missing-msgno", "boolean-msgno"] +) +def test_conversion_rejects_ambiguous_identity_before_writing( + tmp_path: Path, invalid: str +) -> None: + create("sqlite", tmp_path / "target", area_code="TEST") + record = { + "type": "message", + "source": {"format": "msg", "base": "source"}, + "message": {**json.loads(message()), "msgno": 20}, + } + data = json.loads(json.dumps(record)) + if invalid == "missing-source": + data.pop("source") + elif invalid == "missing-msgno": + data["message"].pop("msgno") + elif invalid == "boolean-msgno": + data["message"]["msgno"] = True + with pytest.raises(ToolError) as caught: + list( + write( + "sqlite", + tmp_path / "target", + [json.dumps(row).encode() for row in (record, data)], + ) + ) + assert caught.value.code == "input.structure" + assert caught.value.input_record == 2 + assert caught.value.committed == 0 + assert list(export("sqlite", tmp_path / "target")) == [] + + +def test_mixed_write_keeps_flat_links_in_target_numbering(tmp_path: Path) -> None: + target = tmp_path / "target" + create("sqlite", target, area_code="TEST") + list(write("sqlite", target, [message(subject="existing")])) + envelope = { + "type": "message", + "source": {"format": "msg", "base": "source"}, + "message": {**json.loads(message()), "msgno": 20, "reply_to_msgno": 99}, + } + list( + write( + "sqlite", target, [json.dumps(envelope).encode(), message(reply_to_msgno=1)] + ) + ) + converted = read("sqlite", target, 2) + flat = read("sqlite", target, 3) + assert isinstance(converted, dict) and isinstance(flat, dict) + assert cast(ParsedMessage, converted["message"]).reply_to_msgno is None + assert cast(ParsedMessage, flat["message"]).reply_to_msgno == 1 + + def test_create_write_read_roundtrip( tmp_path: Path, capsys: pytest.CaptureFixture[str] ) -> None: diff --git a/write.html b/write.html index 8a95647..0e7cc64 100644 --- a/write.html +++ b/write.html @@ -24,9 +24,9 @@
On this page
-

JSON in. One commit each.

ftnt write

Append messages from JSON.

One object, or a JSONL batch. Validation finishes before the first append.

00

Usage

ftnt write BASE --format FORMAT [options]

Reads one JSON object from stdin and appends it. --jsonl reads one JSON value per line and appends each value as its own message. Structural checks finish before the writer opens. Each successful append commits on its own and prints a receipt.

append one message
ftnt write ./fixtures/general --format jam < examples/message.json
+
JSON in. One commit each.
ftnt write
Append messages from JSON.
One object, or a JSONL batch. Validation finishes before the first append.
00
Usage
ftnt write BASE --format FORMAT [options]
Reads one JSON object from stdin and appends it. --jsonl reads one JSON value per line and appends each value as its own message. Structural checks finish before the writer opens. Each successful append commits on its own. Export-envelope receipts follow reply-link updates.
append one message
ftnt write ./fixtures/general --format jam < examples/message.json
 
append a batch
ftnt write ./fixtures/general --format jam --jsonl < examples/messages.jsonl
-
Options:
OptionDefaultMeaning
--jsonloffRead a JSONL batch. Without it, the whole stdin value is one object.
--encodingCP850Charset the writer stores.
--lock-timeout5Seconds to wait for the lock. A finite number, zero or greater.
--boardrequired for HudsonHudson board, 1..200. Rejected for every other format.
01
Input object
Required strings: from_name, to_name, subject, body_text. Empty strings pass the structural check; the format writer may still reject them.
FieldJSONWhen omitted
external_idstring or nullnull. This is an external MSGID, not a message number in the base.
from_address, to_addressFTN address string or nullnull. Example: 2:236/77.1@fidonet. Abbreviated forms are not expanded.
posted_atdatetime string or nullnull. Omitted dates are not filled with the current time.
attributes_rawinteger ≥ 0 or nullnull
control_linesarray of objectsempty. Each object needs string name and value. Optional raw defaults to an empty string.
reply_to_msgno, reply1st_msgno, reply_next_msgnointeger ≥ 0 or nullnull
reply_listarray of integers ≥ 0empty
routing_seen_by, routing_patharray of stringsempty. Order and duplicates are kept.
Unknown keys, duplicate keys, NaN, Infinity, a BOM and invalid UTF-8 are rejected. Booleans are not integers. msgno, identity, revision and export envelopes are not write input. Append assigns the message number. There is no automatic reply-link resolver.
examples/message.json
{
+
Options:
OptionDefaultMeaning
--jsonloffRead a JSONL batch. Without it, the whole stdin value is one object.
--encodingCP850Charset the writer stores.
--lock-timeout5Seconds to wait for the lock. A finite number, zero or greater.
--boardrequired for HudsonHudson board, 1..200. Rejected for every other format.
01
Input object
Required strings: from_name, to_name, subject, body_text. Empty strings pass the structural check; the format writer may still reject them.
FieldJSONWhen omitted
external_idstring or nullnull. This is an external MSGID, not a message number in the base.
from_address, to_addressFTN address string or nullnull. Example: 2:236/77.1@fidonet. Abbreviated forms are not expanded.
posted_atdatetime string or nullnull. Omitted dates are not filled with the current time.
attributes_rawinteger ≥ 0 or nullnull
control_linesarray of objectsempty. Each object needs string name and value. Optional raw defaults to an empty string.
reply_to_msgno, reply1st_msgno, reply_next_msgnointeger ≥ 0 or nullnull
reply_listarray of integers ≥ 0empty
routing_seen_by, routing_patharray of stringsempty. Order and duplicates are kept.
Unknown keys, duplicate keys, NaN, Infinity, a BOM and invalid UTF-8 are rejected. Booleans are not integers. Flat write objects cannot include msgno, identity or revision. Append assigns the message number. Flat objects keep their destination-local reply numbers.
Export envelopes are also valid input. Their numeric reply links are remapped within the same source base, board and area in the input batch, including forward links. Links to omitted messages are dropped. Source format, base and message number are required; duplicate source identities are rejected before writing.
examples/message.json
{
   "from_name": "Alice",
   "to_name": "Bob",
   "subject": "Encoding fixture: æøå",
@@ -46,17 +46,21 @@
   "external_id": "2:236/77 fixture-001",
   "from_address": "2:236/77",
   "to_address": "2:236/78",
-  "posted_at": "2026-01-01T12:00:00+00:00",
+  "posted_at": "2026-01-01T12:00:00Z",
   "attributes_raw": 0,
   "control_lines": [
     {
       "name": "CHRS",
       "value": "CP850 2"
+    },
+    {
+      "name": "TZUTC",
+      "value": "+0000"
     }
   ]
 }
 
02
Receipts and reruns
A receipt is one JSON object per committed append:
write_result
{"schema_version":1,"type":"write_result","input_record":1,"identity":{"format":"jam","base":"/resolved/fixtures/general","msgno":1,"board":null},"revision":{"identity":{"format":"jam","base":"/resolved/fixtures/general","msgno":1,"board":null},"location":[0,0],"digest":"opaque"}}
-
input_record counts from 1 in the batch. identity and revision describe what the writer stored. Treat location and digest as opaque. They belong to that base, not to a portable id.
If record 3 fails after record 1 and 2 committed, those two messages stay. Stderr names the failing record and the committed count. A broken pipe can hide a receipt for a message that is already stored. Running the same batch again can append duplicates.

+

input_record counts from 1 in the batch. identity and revision describe what the writer stored. Treat location and digest as opaque. They belong to that base, not to a portable id.

If record 3 fails after record 1 and 2 committed, those two messages stay. Stderr names the failing record and the committed count. A broken pipe can hide a receipt for a message that is already stored. Running the same batch again can append duplicates.

Conversion first appends messages without numeric links, then updates links using the assigned numbers. Receipts contain the revisions after those updates. An interrupted conversion or failed link update can leave committed messages with incomplete links. The error reports the failing record and committed append count.

From 26422d9a442d295ffb8e46c315b496fff2b421f7 Mon Sep 17 00:00:00 2001 From: Odinn Adalsteinsson Date: Sun, 25 Oct 2026 18:37:28 +0100 Subject: [PATCH 2/4] =?UTF-8?q?chore:=20klarg=C3=B8r=20udgivelse=201.1.1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Opdatér pakkeversion, CLI-kontrol, changelog og manual til patchudgivelsen. Bevar Linux/macOS som udgivelsens platformsløfte. --- .github/workflows/publish.yml | 2 +- CHANGELOG.md | 2 +- README.md | 3 ++- catalog.html | 4 ++-- create.html | 4 ++-- decode.html | 4 ++-- docs/release.md | 2 +- export.html | 4 ++-- heads.html | 4 ++-- index.html | 8 ++++---- pyproject.toml | 2 +- read.html | 4 ++-- repair.html | 4 ++-- scripts/build_reading_pages.py | 11 ++++++----- src/golded_ftn_tools/__init__.py | 2 +- tests/test_cli.py | 2 +- write.html | 4 ++-- 17 files changed, 34 insertions(+), 32 deletions(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 6a67332..da4f9e9 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -5,7 +5,7 @@ on: tag: description: Reviewed GitHub release tag required: true - default: v1.1.0 + default: v1.1.1 type: string permissions: contents: read diff --git a/CHANGELOG.md b/CHANGELOG.md index eb3e3e8..3039e9f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # Changelog -## Unreleased +## 1.1.1 — 2026-10-25 - Remap numeric reply links when importing export envelopes, including forward links and existing destinations. Drop links to messages outside the batch and return revisions after link updates. diff --git a/README.md b/README.md index e67b17e..b208d19 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,8 @@ Command-line tools for FTN message bases, JSON fixtures, packets, nodelists and node operations. Python 3.12+. MIT licensed. -Version **1.1.0** adds SQLite and network operations. Packet, nodelist and node commands require the +Version **1.1.1** fixes numeric reply links during conversion. SQLite and network +operations were added in 1.1.0. Packet, nodelist and node commands require the optional `[network]` extra. ```sh diff --git a/catalog.html b/catalog.html index f1c3142..589b7f3 100644 --- a/catalog.html +++ b/catalog.html @@ -18,7 +18,7 @@
FTNT💥Library docs ↗
-
GOLDED / FTN TOOLSRELEASED 1.0.2 · PYTHON 3.12+
+
GOLDED / FTN TOOLSRELEASED 1.1.1 · PYTHON 3.12+
-
GoldED FTN tools / GoldED.devRELEASED 1.0.2 · MIT LICENSE
+
GoldED FTN tools / GoldED.devRELEASED 1.1.1 · MIT LICENSE