Skip to content

bkpr-listbalances takes lightningd down when an account's debits exceed its credits #9487

Description

@ksedgwic

bkpr-listbalances calls plugin_err when an account's debit sum exceeds its credit sum. The bookkeeper is an important plugin, so lightningd shuts down with it.

  1. An account's ledger goes inconsistent. Here: a closed channel whose debits exceed its credits by 12,551,175 msat. Cause not established; an onchain fee counted twice is the likely culprit.
  2. No other bkpr command checks the sums, so nothing notices.
  3. The operator runs bkpr-listbalances.
  4. do_list_balances fails amount_msat_sub for that account and calls plugin_err.
  5. lightningd logs "Killing plugin: exited during normal operation" then "Plugin marked as important, shutting down lightningd!" and exits.

This is the third distinct accounting bug to reach this exit. #5557 (lease fee counted twice), #6534 (negative balance on signet) and #8549 / #8582 (duplicate chain_moves after the accounts.db migration, fixed in #8574) were each closed by fixing that instance's accounting cause, and the exit stayed. This node runs v26.06.7, which includes the #8574 dedup, so the cause here is yet another one. 29e0a1d turned the same kind of assertion in find_txo_chain into a BROKEN log; this check should log and continue as well, whatever the next cause turns out to be. PR to follow.

Hit on my node, ledger going back to 2024. The check is unchanged on master. Workaround until then: don't call bkpr-listbalances. The account's listchainmoves / listchannelmoves events are available on request.


plugin-bookkeeper: Account balance underflow for account 906c4056... (credit 25339109661msat, debit 25351660836msat)
plugin-bookkeeper: Killing plugin: exited during normal operation
Plugin marked as important, shutting down lightningd!

Related: #5557, #6534, #8549, #8582, #8574

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions