Version 2 release

.env

@@ -1,3 +1,42 @@
 SERIAL_PORT='/dev/ttyXRUSB0'
 MODBUS_TCP_GW_IP='192.168.1.140'
 MODBUS_TCP_GW_PORT='8899'
+MOSQUITTO_IP=mosquitto.hs.mfis.net
+MOSQUITTO_PORT=1883
+CERBO_AC_OUT_TOPIC=N/48e7da878d35/vebus/276/Ac/Out
+CERBO_AC_ACTIVEIN_TOPIC=N/48e7da878d35/vebus/276/Ac/ActiveIn
+# Enable a virtual Maxem solar meter slave (default address 001) fed from Cerbo MQTT PV topics.
+CERBO_ENABLE_PV_SLAVE=1
+# RTU slave address for the virtual PV meter used by Maxem autoconfig (typically 001).
+CERBO_PV_TARGET_SLAVE=1
+# Comma-separated PV power topics (W). Values are summed into realtime total PV production.
+CERBO_PV_TOPICS=N/48e7da878d35/solarcharger/283/Pv/0/P,N/48e7da878d35/solarcharger/282/Pv/0/P,N/48e7da878d35/solarcharger/282/Pv/1/P
+# How to fill phase power registers (0x5B16..0x5B1B):
+# - activein: use Cerbo Ac/ActiveIn L1/L2/L3 watts (can go negative on export).
+# - acout: derive watts from ABB phase voltage * Cerbo Ac/Out phase current (non-negative, often best for fuse/load behavior).
+# - abb: keep ABB phase power words unchanged (passthrough).
+CERBO_PHASE_POWER_SOURCE=activein
+# When 1, enforce non-negative phase-power writes:
+# - activein mode: first net exports against imports across phases, then clamp to >=0.
+# - other modes: clamp each phase to >=0.
+CERBO_FORCE_NONNEGATIVE_PHASE_POWER=1
+# When 1, publish MQTT snapshot only after a coherent 3-phase frame is seen for both ActiveIn and Ac/Out.
+CERBO_COHERENT_PHASE_FRAMES=1
+# Max allowed L1/L2/L3 timestamp skew (seconds) inside a coherent frame; frame is skipped if exceeded.
+CERBO_COHERENT_PHASE_FRAME_MAX_SKEW_SECONDS=1.5
+
+SYNTHETIC_HOME_STATE_FILE=/tmp/modbus-softsplit-state.json
+
+## Feature flags 
+# 1 = default to dry-run mode (no RTU open, preview logs only); 0 = normal live mode.
+DRY_RUN_MAXEM_HOME=0
+# INFO by default; use DEBUG to print ABB/Cerbo preview lines.
+LOG_LEVEL=DEBUG
+# 1 enables verbose paho MQTT protocol wire logs (CONNECT/SUBSCRIBE/PUBLISH). Keep 0 for readable DEBUG logs.
+CERBO_MQTT_PROTOCOL_DEBUG=0
+# DEBUG snapshot spam control: 0 disables per-snapshot debug lines; >0 logs at most once per interval in seconds.
+CERBO_MQTT_SNAPSHOT_DEBUG_INTERVAL_SECONDS=0
+# Suppress only known one-byte RTU noise from modbus_tk logs (no protocol behavior change).
+SUPPRESS_SHORT_RTU_REQUEST_LOGS=1
+# Loop heartbeat interval in seconds (0 disables heartbeats).
+STATUS_LOG_INTERVAL_SECONDS=30

.gitignore

@@ -1,3 +1,5 @@
+.secrets-agents
+mcp.json
 *.mod
 
 # Byte-compiled / optimized / DLL files
@@ -16,7 +18,6 @@ dist/
 downloads/
 eggs/
 .eggs/
-lib/
 lib64/
 parts/
 sdist/

.mcp.json

@@ -0,0 +1,13 @@
+{
+  "mcpServers": {
+    "cortex-nexus-gateway": {
+      "command": "npx",
+      "args": [
+        "-y",
+        "mcp-remote",
+        "http://127.0.0.1:18000/mcp",
+        "--allow-http"
+      ]
+    }
+  }
+}

AGENTS.md

@@ -0,0 +1,19 @@
+This repository uses the Documentation and Memory Hygiene policy in
+[`docs/policies/documentation-and-memory-hygiene.md`](docs/policies/documentation-and-memory-hygiene.md).
+
+When a project has a root `.mcp.json` of `mcp.json` file, treat that policy as active.
+
+Read before meaningful work:
+
+- `AGENTS.md`
+- `docs/policies/documentation-and-memory-hygiene.md`
+- relevant project docs such as `README.md`, `docs/`, `docs/adr/`, `docs/archive/`, and `runbooks/`
+
+Follow the policy when updating code, docs, configuration, or durable memory.
+
+For repository-specific durable knowledge, write or update the matching
+document under `docs/` in the same task and treat it as the canonical record. If an appropriate doc does not yet exist, create it.  This is YOUR memory and you can manage it in the best way you yourself can utilize it in the future. 
+Use external memory only as a supplemental reminder, not as the sole source of
+truth for this repository. 
+
+Extra tools, skills, personas, roles, and access to third party platforms is available to you via the MCP servers listed in `*mcp.json` file in the project root. If a connector is not configured it is probably exposed to you via these MCP servers. Use them.

README.md

@@ -24,6 +24,9 @@ the two "virtual" Slave/Client devices, in essence, create 2 new data buses (one
 and allow each of the physical Master/Server devices to have exclusive communication to the "virtual" slaves on these 2 
 new "virtual" buses.
 
+This architecture is intentionally generic and can be adapted for other Modbus device families; ABB meter mapping is the
+current implementation profile, not a hard project limitation.
+
 ![screenshot](/layout.png?raw=true)
 
 ## Notes on this implementation:
@@ -34,8 +37,82 @@ Energy hardware while the other exposes itself via modbus-rtu by way of a USB to
 mentioned earlier does not support modbus-tcp so it was necessary to wire it directly and provide access to the 
 virtual client devices via modbus-rtu using the USB to RS-485 converter.  
 - This code should be easily read and converted in the situation where you want to have multiple RTU based Server devices
-or all Server devices support modbus-tcp.  The easy to use modbus-tk project makes it quite easy to quickly adjust to 
+or all Server devices support modbus-tcp.  The easy to use modbus-tk project makes it quite easy to quickly adjust to
 your specific situation. 
+- The Maxem Home rewrite path is opt-in via `--dry-run-maxem-home`. When that flag is set, the service leaves the RTU
+  serial adapter unopened and logs the instantaneous-power rewrite it would apply to `instantaneous_values` instead of
+  writing values into the RTU slave. This keeps the RTU serving path quiet while we inspect the intended rewrite
+  behavior.
+- You can also set `DRY_RUN_MAXEM_HOME=1` in `.env` to make dry-run the default without changing service/unit args.
+- In dry-run mode the Maxem preview is intentionally short and easy to compare against dashboards:
+  `ABB source: X W`, `Cerbo Usage to Maxem: Y W`, and `Cerbo Phase Watts to Maxem: L1=..., L2=..., L3=...`.
+  When PV slave emulation is enabled, dry-run also logs `Cerbo PV to Maxem (slave 001): ...`.
+  These lines are emitted at `DEBUG` level (set `LOG_LEVEL=DEBUG` when you want them).
+- The active rewrite story now reads directly from Victron CerboGX MQTT (read-only) and rewrites selected words in
+  ABB `instantaneous_values` while mirroring all other words verbatim.
+  - `active_power_total` (`0x5B14/0x5B15`) is sourced from `Ac/ActiveIn` total watts.
+  - `active_power_l1/l2/l3` (`0x5B16..0x5B1B`) follow `CERBO_PHASE_POWER_SOURCE`.
+  - `current_l1/l2/l3/n` (`0x5B0C..0x5B13`) are sourced from `Ac/Out` phase currents.
+  - `Ac/ActiveIn` values may be positive or negative; `Ac/Out` currents are clamped to non-negative values.
+  - `CERBO_PHASE_POWER_SOURCE` can pivot phase-power behavior without code edits:
+    - `activein` uses Cerbo `Ac/ActiveIn` per-phase watts.
+      When `CERBO_FORCE_NONNEGATIVE_PHASE_POWER=1`, exports are first netted against imports across phases, then clamped to `>=0`.
+    - `acout` derives phase watts from ABB phase voltages and Cerbo `Ac/Out` phase currents.
+    - `abb` leaves phase-power words unchanged from ABB passthrough.
+  - `CERBO_COHERENT_PHASE_FRAMES=1` publishes snapshots only after complete 3-phase updates for both
+    `Ac/ActiveIn` and `Ac/Out`, reducing mixed-time phase combinations.
+  - Optional virtual PV meter emulation for Maxem slave `001` is available via:
+    - `CERBO_ENABLE_PV_SLAVE=1`
+    - `CERBO_PV_TARGET_SLAVE=1`
+    - `CERBO_PV_TOPICS=<comma-separated topic list>`
+    The PV total is summed from those topics and written to the `instantaneous_values` active-power words on slave `001`.
+    Phase power/current words for slave `001` are synthesized coherently from that total (equal split by phase, amps from ABB phase voltage).
+  This keeps Maxem home/grid power semantics aligned with grid import/export while preserving AC-out current safety inputs
+  used for EV phase protection.
+- The dry-run logger prints one semantic line before the preview values so it is obvious that the preview is the
+  ABB instantaneous register block being rewritten from Cerbo MQTT.
+- Register bundle captures and replay previews live under `tools/`. The dump helper defaults to `instantaneous_values`
+  only, and the replay helper prints the same preview lines without touching the RTU adapter.
+- Preview semantics are simple:
+  - `ABB source` is the decoded instantaneous active-power total from ABB.
+  - `Cerbo Usage to Maxem` is the rewrite total watts from `Ac/ActiveIn`.
+  - `Cerbo Phase Watts to Maxem` are per-phase rewrite watts from `Ac/ActiveIn`.
+  - `Cerbo Phase Currents to Maxem` are per-phase + neutral current amps from `Ac/Out`.
+- Core application modules now live under `lib/` so the repo root stays focused on the entrypoint, tools,
+  and docs.
+
+## Development workflow
+
+- Test files use numbered prefixes like `tests/10_test_maxem_home_usage.py` and `tests/20_test_dump_and_replay.py` so
+  related suites can grow in a predictable order.
+- `pytest.ini` is configured to collect only numbered test files from `tests/`.
+- The quickest local checks are `python3 -m pytest` and `python3 -m py_compile` on the touched modules.
+- For the register tooling, run `python3 tools/dump_register_block.py --help` and
+  `python3 tools/replay_maxem_preview.py --help` before using real captures.
+- `python3 tools/dump_register_block.py` defaults to the ABB `instantaneous_values` block; pass `--register` to add
+  extra blocks only when you truly need them.
+- `python3 tools/replay_maxem_preview.py --bundle <file>` prints the same short preview line format the dry-run
+  runtime uses (`ABB source` plus rewrite lines) from a captured bundle.
+- `python3 tools/inspect_instantaneous_payload.py --bundle <file>` unpacks the key ABB instantaneous fields and shows
+  source vs rewritten values plus the exact word addresses that changed.
+- `python3 main.py --trace-instantaneous-payload` enables the same field-level diff in live runtime logs so we can
+  verify exactly what is being written without changing default behavior.
+- Cerbo MQTT poller is read-only and subscribes to:
+  - `CERBO_AC_OUT_TOPIC` (default `N/48e7da878d35/vebus/276/Ac/Out`)
+  - `CERBO_AC_ACTIVEIN_TOPIC` (default `N/48e7da878d35/vebus/276/Ac/ActiveIn`)
+  - `CERBO_PV_TOPICS` (default: three configured `solarcharger/.../Pv/.../P` topics; summed for virtual PV meter power)
+- At `DEBUG` level the runtime logs snapshot updates from MQTT and preview lines (`ABB source` / `Cerbo ... to Maxem`).
+- To keep DEBUG readable by default:
+  - `CERBO_MQTT_PROTOCOL_DEBUG=0` suppresses raw paho wire logs (`Sending CONNECT`, `Received PUBLISH`, etc).
+  - `CERBO_MQTT_SNAPSHOT_DEBUG_INTERVAL_SECONDS=0` suppresses per-message snapshot debug spam.
+  - Set `CERBO_MQTT_PROTOCOL_DEBUG=1` and/or `CERBO_MQTT_SNAPSHOT_DEBUG_INTERVAL_SECONDS=<seconds>` only for deep MQTT troubleshooting.
+- Startup logs print effective Cerbo source settings and where values came from (`env`, `.env`, or defaults).
+- The main loop now handles `Ctrl-C` cleanly in one interrupt and stops the poller and servers without a traceback.
+- Canonical project-specific runtime rules and durable decisions live in
+  `docs/decisions/project-conventions.md`.
+- Long-run logging/observability tuning is controlled with:
+  - `SUPPRESS_SHORT_RTU_REQUEST_LOGS=1` (default) to suppress only `invalid request: Request length is invalid 1`.
+  - `STATUS_LOG_INTERVAL_SECONDS=30` (default) to emit periodic heartbeat summaries instead of per-loop update spam.
 
 ## Tested Hardware
 This has been tested with the Exar USB to RS-485 adapter and with the Waveshare CAN Hat (CANbus and RS-485 add-on) for

docs/decisions/project-conventions.md

@@ -0,0 +1,114 @@
+# Project Conventions
+
+This document is the canonical home for durable, project-specific decisions and
+operator-facing assumptions for `modbus-softsplit`.
+
+## Active Rewrite Path
+
+- The Maxem rewrite path reads source data directly from Victron CerboGX MQTT
+  (read-only), not from Domoticz.
+- MQTT broker defaults:
+  - host `mosquitto.hs.mfis.net`
+  - port `1883`
+  - active-in topic base `N/48e7da878d35/vebus/276/Ac/ActiveIn`
+  - ac-out topic base `N/48e7da878d35/vebus/276/Ac/Out`
+  - pv topic list (`CERBO_PV_TOPICS`) defaults to:
+    - `N/48e7da878d35/solarcharger/283/Pv/0/P`
+    - `N/48e7da878d35/solarcharger/282/Pv/0/P`
+    - `N/48e7da878d35/solarcharger/282/Pv/1/P`
+- Power rewrite source:
+  - `Ac/ActiveIn/L1|L2|L3/P` -> active power total + per phase words.
+  - Values may be positive or negative.
+- `CERBO_PHASE_POWER_SOURCE` controls phase-power words (`0x5B16..0x5B1B`):
+  - `activein` -> phase power from Cerbo `Ac/ActiveIn` (signed).
+  - `acout` -> phase power derived from ABB phase voltages and Cerbo `Ac/Out` currents.
+  - `abb` -> phase power passthrough from ABB (no rewrite on phase words).
+- `CERBO_FORCE_NONNEGATIVE_PHASE_POWER=1` enforces non-negative phase-power writes:
+  - in `activein` mode, export is netted against import across phases first, then clamped to `>=0`.
+  - in other modes, each phase is clamped independently to `>=0`.
+- Current rewrite source:
+  - `Ac/Out/L1|L2|L3/I` -> phase current words.
+  - `Ac/Out/N/I` -> neutral current word when present.
+  - AC-out currents are clamped to `>= 0` before encoding.
+- `CERBO_COHERENT_PHASE_FRAMES=1` requires full 3-phase refresh for both power and current
+  inputs before publishing a new rewrite snapshot.
+- `CERBO_COHERENT_PHASE_FRAME_MAX_SKEW_SECONDS` bounds acceptable timestamp skew across
+  phase updates to reduce mixed-time phase combinations.
+- The ABB target register block is `instantaneous_values`.
+- Rewritten words in that block are:
+  - `0x5B0C/0x5B0D`, `0x5B0E/0x5B0F`, `0x5B10/0x5B11`, `0x5B12/0x5B13`
+    current L1/L2/L3/N from Cerbo `Ac/Out`.
+  - `0x5B14/0x5B15` active power total from Cerbo `Ac/ActiveIn`.
+  - `0x5B16/0x5B17`, `0x5B18/0x5B19`, `0x5B1A/0x5B1B`
+    active power per-phase from `CERBO_PHASE_POWER_SOURCE`.
+- All other words in `instantaneous_values` and all other Maxem register blocks
+  are mirrored unchanged from the ABB source.
+- Optional PV meter emulation (`CERBO_ENABLE_PV_SLAVE=1`) publishes a virtual
+  Maxem-compatible slave (default address `001` via `CERBO_PV_TARGET_SLAVE`):
+  - all Maxem register blocks mirror ABB source values by default.
+  - in `instantaneous_values`, slave `001` rewrites:
+    - `0x5B14/0x5B15` to summed PV watts from `CERBO_PV_TOPICS`.
+    - `0x5B16..0x5B1B` to an equal 3-phase split of that total.
+    - `0x5B0C..0x5B13` to derived non-negative phase currents from
+      rewritten phase watts and ABB phase voltages; neutral current is `0`.
+- Preview logs should stay short and verifiable:
+  - `ABB source: X W`
+  - `Cerbo Usage to Maxem: Y W`
+  - `Cerbo Phase Watts to Maxem: L1=..., L2=..., L3=...`
+  - `Cerbo Phase Currents to Maxem: L1=..., L2=..., L3=..., N=...`
+- These preview lines are logged at `DEBUG` level and are intended as opt-in
+  operator diagnostics (`LOG_LEVEL=DEBUG`).
+- The live RTU path should emit the same short preview when the instantaneous
+  rewrite value changes so operators can verify the actual write path without
+  enabling dry-run mode.
+
+## Register Tooling
+
+- `tools/dump_register_block.py` defaults to `instantaneous_values` only.
+- `tools/replay_maxem_preview.py` replays the instantaneous-power preview from a
+  captured bundle.
+- `tools/inspect_instantaneous_payload.py` is the deep-dive tool for unpacking
+  ABB source vs rewritten payload values and showing exactly which
+  `instantaneous_values` words changed.
+- `docs/runbooks/maxem-live-validation.md` is the canonical operational
+  checklist for long-running live validation.
+- `docs/Maxem MX Home 4 handleiding.pdf` is installation/wiring guidance and
+  useful for system behavior context, but does not replace ABB register mapping
+  references.
+- `total_accumulators` is legacy prototype output and is not part of the active
+  Maxem rewrite story.
+
+## Runtime Guardrails
+
+- `--dry-run-maxem-home` is opt-in and must not open the RTU serial adapter.
+- `DRY_RUN_MAXEM_HOME=1` in `.env` is the default-mode toggle equivalent to
+  starting with `--dry-run-maxem-home`.
+- `--trace-instantaneous-payload` is opt-in and intended for diagnostics only.
+- Cerbo MQTT subscriptions must remain read-only; do not publish/control from
+  this runtime.
+- MQTT callbacks must stay lightweight and non-blocking; RTU serving path must
+  remain timing-safe.
+- Startup should log effective Cerbo MQTT source settings and source precedence
+  (`env` vs `.env` vs defaults).
+- Startup should log effective PV slave settings:
+  `CERBO_ENABLE_PV_SLAVE`, `CERBO_PV_TARGET_SLAVE`, and `CERBO_PV_TOPICS`.
+- `CERBO_MQTT_PROTOCOL_DEBUG=0` should remain default so DEBUG logs stay operator-readable; enable only during MQTT wire troubleshooting.
+- `CERBO_MQTT_SNAPSHOT_DEBUG_INTERVAL_SECONDS=0` should remain default to prevent per-message snapshot log flooding.
+- The serving loop should remain timing-safe for the RTU client.
+- High-volume loop status logs should be periodic instead of per-cycle to avoid
+  unnecessary log I/O overhead (`STATUS_LOG_INTERVAL_SECONDS`, default `30`).
+- RTU protocol validation must remain strict; do not reinterpret malformed
+  short frames as valid requests.
+- `SUPPRESS_SHORT_RTU_REQUEST_LOGS=1` (default) suppresses only the specific
+  known-noise line `invalid request: Request length is invalid 1` at logging
+  time, without changing RTU frame parsing behavior.
+
+## Documentation and Memory Hygiene
+
+- Project-specific durable knowledge belongs in `docs/` first.
+- Prefer `docs/decisions/` for stable implementation rules and `docs/runbooks/`
+  for operational procedures.
+- External memory notes are supplemental and should not be the only durable
+  record for this repository.
+- When project behavior changes, update the relevant repo docs in the same task
+  so future runs do not need to infer intent from chat history.