read_projection raises 42704 after any rewrite: projection rows keep the retired storage id

[OffgridwithJD]

Summary

Any rewrite that mints a new storage id leaves pgcolumnar.projection keyed to the old one. pgcolumnar.read_projection resolves the table's current storage id, finds no projection under it, and raises 42704 — for a projection that is still declared and whose base table is intact. Nothing re-records the rows, nothing warns, and the old rows stay behind.

Pre-existing on main. Filed at jdatcmd's request out of the #867 review, where I raised it as a regression and it turned out not to be one: #867 introduces none of this, and the same failure is on main unpatched.

Reproduction, on main 4c024d0, pg18a

CREATE TABLE pk (id int, a int, b text) USING pgcolumnar;
SELECT pgcolumnar.add_projection('pk','pkp',ARRAY['a','b'],ARRAY['a']);
INSERT INTO pk SELECT g, g%50, 'b'||g FROM generate_series(1,5000) g;

                          storage id     pgcolumnar.projection names   read_projection
after first write        10000000000     10000000000/0/base
                                         10000000000/1/pkp             5000
TRUNCATE + re-INSERT     10000000002     10000000000                   ERROR 42704
pgcolumnar.vacuum('pk')  10000000003     10000000000                   ERROR 42704

ERROR: projection "pkp" does not exist on "pk". The base table holds its 5000 rows throughout, and pgcolumnar.projection_declaration still holds pk/pkp — the declaration is never destroyed.

Mechanism

pgcolumnar_relation_set_new_filelocator mints a new id unconditionally on every rewrite (src/columnar_tableam.c, storageId = PgColumnarNextStorageId(); followed by PgColumnarWriteNewMetapage). read_projection reads storageId = PgColumnarStorageId(rel) and calls PgColumnarListProjections(storageId), so it looks under the current id only, and raises ERRCODE_UNDEFINED_OBJECT when the name is not in that list (src/columnar_projection.c).

It is recoverable, and that is the part worth acting on

pgcolumnar.rebuild_projections() fixes it:

rebuild_projections()        returns 1
read_projection after that   5000
pgcolumnar.projection names  10000000000  10000000003

So the data is never lost — the declaration is enough to rebuild. Two things follow. A user whose projection has silently stopped answering has no way to learn that one manual call fixes it; nothing in the error text, the docs, or pgcolumnar.stats points at rebuild_projections. And the rebuild does not remove the rows under the retired id — 10000000000 is still there afterwards, so the orphans accumulate per rewrite. That second half is what #867 closes for the TRUNCATE path specifically.

What I did not establish

src/columnar_vacuum.c:1343-1362 and :1524-1543 do re-record projections under newStorageId. I predicted pgcolumnar.vacuum('pk') would therefore restore the projection and it did not — the measurement above shows the rows still naming 10000000000 after it. I have not worked out whether those paths are simply not reached on this fixture or whether something else is wrong, and I am not going to guess. Anyone picking this up should start there: two code paths visibly do the right thing and the observable behaviour says the right thing did not happen.

Suggested shape

Whatever fixes it should hold three properties, none of which any suite asserts today:

1. After any rewrite, read_projection answers as it did before the rewrite.
2. No pgcolumnar.projection row names a storage id the table no longer has.
3. The declaration survives, so a rebuild is always possible.

test/truncate_cleanup.sh on #867 asserts 2 and 3 for TRUNCATE. Nothing asserts 1 anywhere, which is why this has been silent.

Measured on pg18a, .so d2e0847f3173, tree 4c024d0.