Skip to content

query: --prune-inwit watermark, NETWORK_LIMITED, Core -8 - #654

Open
rearden-grok[bot] wants to merge 17 commits into
masterfrom
store/inwit-prune
Open

rearden-grok[bot] wants to merge 17 commits into
masterfrom
store/inwit-prune

Conversation

@rearden-grok

@rearden-grok rearden-grok Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Plan 09. Own store stack on master. #659 (store/txfixed) sits on this branch.

Rebased onto master (698177c8).

Unpruned nodes read inwit.body. --prune-inwit stops serving witness at or below a height watermark and keeps the last 288 heights as store/inwit.window/{height}.bin plus a RAM cache.

  • {store}/inwit.prune is a 4-byte LE sidecar. File presence is prune mode; u32::MAX means nothing dropped yet. Once the tip is above 288, confirm writes pruneheight = tip - 288.
  • Reconstruct, disconnect, and wire loads return StoreError::Pruned at or below the watermark.
  • --prune-inwit advertises NETWORK_LIMITED|WITNESS|P2P_V2. getdata maps pruned blocks to notfound. RPC uses Core -8; Electrum transaction.get returns "pruned"; Esplora omits the vin key on pruned txs.
  • --prune-inwit-ram-threshold-bytes defaults to 256 MiB. 0 keeps nothing in RAM: every height, including tiny IBD blocks, is read from its file. Disconnect and a re-confirm of the same height drop that height's RAM rows before the new rows are inserted.
  • Enabling prune on an existing archive seeds the window. A pruned datadir without --prune-inwit is a hard startup error. NixOS pruneInwit. No SCHEMA_VERSION bump.

Closes #589

Test plan

  • cargo test -p rbitcoin-query prune_
  • cargo test -p rbitcoin-query prune_ram_threshold_zero_spills_tiny_blocks
  • cargo test -p rbitcoin-node prune_inwit_is_kebab
  • required CI

Do not merge unless asked.

Comment thread crates/rbitcoin-query/src/lib.rs Fixed
Comment thread crates/rbitcoin-query/src/lib.rs Fixed
@rearden-grok
rearden-grok Bot force-pushed the store/inwit-prune branch 2 times, most recently from badd49c to 0cbf383 Compare September 20, 2026 22:31
@rearden-grok
rearden-grok Bot changed the base branch from net/cjdns to master September 20, 2026 22:31
@rearden-grok
rearden-grok Bot force-pushed the store/inwit-prune branch 3 times, most recently from b05b4d7 to cca1b03 Compare September 22, 2026 03:45
rearden-grok Bot and others added 14 commits September 22, 2026 09:04
Wire reconstruct must not look like Corrupt when inwit was dropped on
purpose. Query holds a stub pruneheight; rolling segments come next.

Co-authored-by: Cursor <cursoragent@cursor.com>
A reorg through pruned inwit cannot rebuild vin from spent alone. Fail
closed with StoreError::Pruned and leave the tip in place.

Co-authored-by: Cursor <cursoragent@cursor.com>
`--prune-inwit` must survive reopen. File presence is prune mode;
u32::MAX means nothing dropped yet. Confirm past 288 heights writes
tip-288 so the last BIP159 window stays reconstructable.

Co-authored-by: Cursor <cursoragent@cursor.com>
A pruned node is not NETWORK. VERSION, addrv2, and getaddr use the hub
flag; getdata maps StoreError::Pruned to notfound instead of disconnect.

Co-authored-by: Cursor <cursoragent@cursor.com>
getblock/getrawtransaction/getblockstats use the same -8 message Core
prints for pruned data. Electrum transaction.get returns \"pruned\";
Esplora answers 404.

Co-authored-by: Cursor <cursoragent@cursor.com>
Kebab CLI / snake conf / NixOS camelCase. Startup sets Query prune
mode and PeerHub so IBD from genesis advertises NETWORK_LIMITED before
the first 288-height watermark.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Prune mode no longer rewrites historical inwit.loc; recent witness lives in a 288-height RAM window with a byte threshold and store/inwit.window spill, and startup refuses a pruned datadir without --prune-inwit.

Co-authored-by: Cursor <cursoragent@cursor.com>
confirm_blocks_run_with_create_pins scored CRAP 31.1 after the RAM-window hook; move that post-tip work to a named helper so coverage-crap stays under 30.

Co-authored-by: Cursor <cursoragent@cursor.com>
Load by matching a dirent whose parsed height is the window slot, not
by joining the path. Intermediate path-join and dirent-only steps are
the same end state.

Co-authored-by: Cursor <cursoragent@cursor.com>
open_inwit hit clippy cognitive_complexity 27/25 after prune-inwit branches; loc open and count-skew repair are named seams, not a silent allow.

Co-authored-by: Cursor <cursoragent@cursor.com>
Disconnect and a later block at the same height left the old creates
in the fk map, so the 288-height cap no longer bounded RAM. Spill
loads open the pinned {height}.bin path, and a failed unlink of a
dropped spill file is an IO error.
A height read from the store was joined into `{height}.bin` and opened
as-is. Canonicalize the path, then require it to stay under `inwit.window`,
so a symlink cannot be read as witness.
`--prune-inwit-ram-threshold-bytes 0` is a supported mode. Confirm still
writes `inwit.window/{height}.bin`, and the RAM window stays empty so tiny
IBD blocks are not kept in process memory.
Unpruned nodes read inwit.body. Pruned nodes serve the last 288 heights
from inwit.window/{height}.bin plus a RAM cache. A zero RAM cap keeps
nothing in memory. Plan 09, OPERATOR, COMPAT, and the CLI no longer
describe unlinking a rolling inwit stem.
The usage string is a format template. A bare {height} does not compile.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant