add _fsm _vm dump by KorsunDaria · Pull Request #43 · df7cb/pg_filedump

KorsunDaria · 2026-08-03T07:03:38Z

Add support for parsing `_fsm` and `_vm` files

Motivation

pg_filedump currently only supports parsing the main fork of heap/index files — there's no human-readable dump support for _vm (visibility map) and _fsm (free space map) files, so diagnosing them requires reading raw bytes by hand.

This also helps with debugging in general:

The free space map shows how full pages are (master and replica).
The visibility map shows the visibility status of pages, which helps explain why a particular query plan was chosen.

What's added

New parsing modes for _fsm and _vm , plus a diff mode to compare two dumps of the same fork.

Flags

`pg_fsm`

Flag	Purpose
`-H`	page headers, including`fp_next_slot`
`-i`	internal tree nodes
`-s`	leaf slots + per-heap-page free-space category
`-a`	all of the above combined
`-q`	summary: per-category (0–255) histogram of heap pages + stats
`--range A-B` / `--page N`	page range
`--extra`	full per-slot/per-node listing instead of the default compressed ranges
`--heap-page N`	locate a specific heap page
`--min N` / `--max N`	filter leaf slots by`avail_bytes` (dump `-s` only)
`--only-changed`	diff mode: show only pages whose value actually changed

`pg_vm`

Flag	Purpose
`-H`	page header dump
`-q`	summary (counts of visible/frozen pages)
`--heap-range A-B`	a heap-page range
`--heap-page N`	look up a single heap page's VM bits
`--expand`	full per-heap-page listing instead of run-length-compressed ranges
`--only-not-visible` / `--only-not-frozen / --notF / --notV`	dump mode: show only pages missing that bit
`--only-changed`	diff mode: show only pages where visible/frozen status differs between two dumps

Example usage

./pg_filedump -F dump -Hs --min 76 16399_fsm 16399_fsm.txt
./pg_filedump -F diff -q 16399_fsm 16399_1_fsm out.txt

./pg_filedump -V dump -a 16399_vm 16399_vm.txt

`pg_fsm`

-H — page headers

$ pg_fsm dump -H /Users/dariakorsun/pgdata1/base/5/16399_fsm
=== pg_fsm dump: /Users/dariakorsun/pgdata1/base/5/16399_fsm ===
file size: 122880 bytes, total pages: 15 (8192 bytes/page)
flags: headers=on internal=off slots=off expand=off
\-- 0 fsm [ROOT]
      header: pd_lsn=0 pd_checksum=0 pd_flags=0x0 pd_lower=24 pd_upper=8192 pd_special=8192 pagesize=8192 layout_version=4 fp_next_slot=0
    \-- 1 fsm [INTERNAL]
          header: pd_lsn=0 pd_checksum=0 pd_flags=0x0 pd_lower=24 pd_upper=8192 pd_special=8192 pagesize=8192 layout_version=4 fp_next_slot=0
        \-- 2 fsm [LEAF]
              header: pd_lsn=0 pd_checksum=0 pd_flags=0x0 pd_lower=24 pd_upper=8192 pd_special=8192 pagesize=8192 layout_version=4 fp_next_slot=4069
              page max: category=0 (0 bytes) at slot -1
              Free space on page: 0 (sum over 4069 slots)

-i — internal tree nodes

\-- 14 fsm [LEAF]
              NONLEAF (4095 nodes):
                  [0-1]                 77
                  [2]                    0
                  [3]                   77
                  [4-6]                  0
                  [7]                   77
                  [8-15]                 0
                  [16]                  77
                  [17-33]                0
                  [34]                  77
                  [35-68]                0
                  [69]                  77
                  [70-139]               0
                  [140]                 77
                  [141-280]              0
                  [281]                 77
                  [282-562]              0
                  [563]                 77
                  [564-1126]             0
                  [1127]                77
                  [1128-2254]            0
                  [2255]                77
                  [2256-4094]            0
              LEAF (4069 nodes):
                  [0-415]                0
                  [416]                 77
                  [417-4068]             0
              page max: category=77 (2464 bytes) at slot 416
              Free space on page: 2464

-s — leaf slots

\-- 14 fsm [LEAF]
              page max: category=77 (2464 bytes) at slot 416
              Free space on page: 2464

              leaf slots (4069, starting at 48828):
                heap page    48828-49243   : category=0   avail_bytes=0     (416 page(s))
                heap page    49244         : category=77  avail_bytes=2464
                heap page    49245-52896   : category=0   avail_bytes=0     (3652 page(s))

-a — combined flags -His

-q — summary

--------------------------------------------------------------
SUMMARY
--------------------------------------------------------------
root pages: 1  internal pages: 1  leaf pages: 13
zero/hole pages: 1  invalid-header pages: 0

Max free-space category: 77 (2464 bytes), page 14 slot 416

Total avail_bytes : 2464

heap pages by category:
  category   0 (    0 bytes): 48827 page(s)
  category  77 ( 2464 bytes): 1 page(s)

--range A-B / --page N- page in range

--extra

\-- 14 fsm [LEAF]
      header: pd_lsn=0 pd_checksum=0 pd_flags=0x0 pd_lower=24 pd_upper=8192 pd_special=8192 pagesize=8192 layout_version=4 fp_next_slot=0
      NONLEAF (4095 node(s), expanded):
          [   0]  77  77   0  77   0   0   0  77   0   0   0   0   0   0   0   0  77   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
          [  32]   0   0  77   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
          [  64]   0   0   0   0   0  77   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
   .
   .
   .

      leaf slots (4069, one per heap page starting at 48828, expanded):
      	  slot    0 (heap page    48828): category=  0 avail_bytes=    0
      	  slot    1 (heap page    48829): category=  0 avail_bytes=    0
      	  slot    2 (heap page    48830): category=  0 avail_bytes=    0
      	  slot    3 (heap page    48831): category=  0 avail_bytes=    0

--heap-page N - jnly one heap page

--min N / --max N

 \-- 14 fsm [LEAF]
              header: pd_lsn=0 pd_checksum=0 pd_flags=0x0 pd_lower=24 pd_upper=8192 pd_special=8192 pagesize=8192 layout_version=4 fp_next_slot=0
              page max: category=77 (2464 bytes) at slot 416
              Free space on page: 2464

              leaf slots (4069, starting at 48828) [filtered by avail_bytes]:
                heap page    49244         : category=77  avail_bytes=2464

diff mode

=== pg_fsm diff ===
old: /Users/dariakorsun/pgdata-dtrace/base/5/24576_2_fsm (3 pages)
new: /Users/dariakorsun/pgdata-dtrace/base/5/24576_fsm (3 pages)

\-- 0 fsm [ROOT] CHANGED
      internal-node    0-1       : 239 -> 218  [CHANGED] (2 node(s))
      internal-node    2         :   0 ->   0  [same]
      internal-node    3         : 239 -> 218  [CHANGED]
      internal-node    4-6       :   0 ->   0  [same] (3 node(s))
      internal-node    7

--only-changed (diff mode)

=== pg_fsm diff ===
old: /Users/dariakorsun/pgdata-dtrace/base/5/24576_2_fsm (3 pages)
new: /Users/dariakorsun/pgdata-dtrace/base/5/24576_fsm (3 pages)
(--only-changed: SAME/empty subtrees and unchanged runs are hidden)

\-- 0 fsm [ROOT] CHANGED
      leaf-slot heap page        0           : 239 -> 218 ( 7648 ->  6976 B)
    \-- 1 fsm [INTERNAL] CHANGED
          leaf-slot heap page        0           : 239 -> 218 ( 7648 ->  6976 B)
        \-- 2 fsm [LEAF] CHANGED
              leaf-slot heap page        0           : 239 -> 218 ( 7648 ->  6976 B)

`pg_vm`

-H — page header dump

=== pg_vm dump: /Users/dariakorsun/pgdata-dtrace/base/5/24579_vm ===
file size: 8192 bytes, total pages: 1, 8192 bytes/page
heap page range covered: [0, 32671]


-- physical page inventory (-H) --
vm page 0: lsn=15F181E78 checksum=0 flags=0x0 lower=24 upper=8192 special=8192

-- heap page status (compressed) --
heap pages        0-862     : visible+frozen (863 page(s))
heap pages      863-32671   : none (31809 page(s))

-q — summary

--------------------------------------------------------------
SUMMARY
--------------------------------------------------------------
heap pages in range: 32672
all-visible: 863  all-frozen: 863  error page(s): 0

--heap-range A-B

--heap-page N

--expand

=== pg_vm dump: /Users/dariakorsun/pgdata-dtrace/base/5/24579_vm ===
file size: 8192 bytes, total pages: 1, 8192 bytes/page
heap page range covered: [0, 32671]


-- physical page inventory (-H) --
vm page 0: lsn=15F181E78 checksum=0 flags=0x0 lower=24 upper=8192 special=8192

-- heap page status (expanded) --
heap page        0: visible+frozen
heap page        1: visible+frozen

--only-not-visible / (--only-not-frozen)

=== pg_vm dump: /Users/dariakorsun/pgdata1/base/5/16399_vm ===
file size: 16384 bytes, total pages: 2, 8192 bytes/page
heap page range covered: [0, 65343]


-- page_headerysical page headers (-H) --
vm page 0: lsn=70304A50 checksum=0 flags=0x0 lower=24 upper=8192 special=8192
vm page 1: lsn=7043FA08 checksum=0 flags=0x0 lower=24 upper=8192 special=8192

-- heap page status (compressed) --
heap pages        0-49244   : visible (49245 page(s))
heap pages    49245-65343   : none (16099 page(s))

(diff mode)

=== pg_vm diff ===
old: /Users/dariakorsun/pgdata-dtrace/base/5/24579_2_vm (1 pages)
new: /Users/dariakorsun/pgdata-dtrace/base/5/24579_vm (1 pages)
heap page range covered: [0, 32671]

heap pages        0-862     : none -> visible+frozen  [CHANGED] (863 page(s))
heap pages      863-32671   : none -> none  [same] (31809 page(s))

--only-changed (diff mode)

=== pg_vm diff ===
old: /Users/dariakorsun/pgdata-dtrace/base/5/24579_2_vm (1 pages)
new: /Users/dariakorsun/pgdata-dtrace/base/5/24579_vm (1 pages)
heap page range covered: [0, 32671]

heap pages        0-862     : none -> visible+frozen  [CHANGED] (863 page(s))

df7cb · 2026-08-03T14:34:33Z

Hmm. Does this add anything over the in-core pg_freespacemap and pg_visibility modules? pg_filedump is mostly meant to be used in situations where the server doesn't even start, and the fsm/vm forks should be entirely optional.

KorsunDaria · 2026-08-04T05:03:37Z

Hello!

On top of pg_freespacemap / pg_visibility:

Finds corrupted pages - print what's wrong with the headers
Header dump - pd_lsn/pd_lower/pd_upper/... decoded into readable fields
Visual tree structure - root/internal/leaf layout is made explicit
Leaf and non-leaf dump - Internal and leaf nodes printed in distinct sections, helping identify the level where something goes wrong
Decodes categories - category -> approximate free bytes
Summary statistics - aggregate counts (visible/frozen, changed pages, avail_bytes totals)

Moreover:

No running server needed
Diff mode - compares FSM/VM files directly; no SQL equivalent exists

Example of where this would help - a recent pgsql-hackers thread [1] describes FSM data on a standby silently diverging from the primary during recovery, causing slow inserts after failover. An offline dump/diff tool like this would make such discrepancies easy to catch and reproduce.

[1] https://www.postgresql.org/message-id/flat/6A6138F4-8D6F-463D-9ECC-38C0881308FE%40yandex-team.ru#ece75f9c21e29c2043d65f7377e096bc

KorsunDaria · 2026-08-11T05:38:14Z

@df7cb, may be you have suggestions for code improvements? or other wishes regarding this pull request?

df7cb · 2026-08-11T18:49:50Z

Hi, sorry for not replying last week. I'm currently on vacation and need to have a deeper look at it when I return.

add _fsm _vm dump

aa11766

KorsunDaria force-pushed the master branch from b5a45d1 to aa11766 Compare August 3, 2026 10:07

KorsunDaria force-pushed the master branch from 86ce9b8 to 28c91a4 Compare August 12, 2026 11:39

fix test errors

9117ed0

KorsunDaria force-pushed the master branch from 62d170c to 9117ed0 Compare August 13, 2026 02:54

Navigation Menu

add _fsm _vm dump - #43

add _fsm _vm dump#43
KorsunDaria wants to merge 2 commits into
df7cb:masterfrom
KorsunDaria:master

KorsunDaria commented Aug 3, 2026

Uh oh!

df7cb commented Aug 3, 2026

Uh oh!

KorsunDaria commented Aug 4, 2026 •

edited

Loading

Uh oh!

KorsunDaria commented Aug 11, 2026

Uh oh!

df7cb commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

KorsunDaria commented Aug 3, 2026

Add support for parsing _fsm and _vm files

Motivation

What's added

Flags

pg_fsm

pg_vm

Example usage

pg_fsm

pg_vm

Uh oh!

df7cb commented Aug 3, 2026

Uh oh!

KorsunDaria commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KorsunDaria commented Aug 11, 2026

Uh oh!

df7cb commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add support for parsing `_fsm` and `_vm` files

`pg_fsm`

`pg_vm`

`pg_fsm`

`pg_vm`

KorsunDaria commented Aug 4, 2026 •

edited

Loading