fix: fan UPDATE rows out to covering projections

CHANGELOG.md

@@ -35,6 +35,28 @@ true until the next version shipped.
 
 ### Fixed
 
+- `UPDATE` now fans the new row version out to every covering projection, so a
+  projection scan stops answering as if the updated rows were gone.
+
+  **A projection stored the base row number, and `UPDATE` never gave it the new
+  one.** A projection joins back to the base table on the row number and takes
+  visibility from the base delete vector, so `DELETE` needs no rewrite: the
+  vector hides the old number from the projection too. `UPDATE` is delete-old
+  plus insert-new, and only the delete half reached the projection. The
+  projection kept the old number, the delete vector hid it, and the new number
+  was nowhere. `read_projection` returned no rows, and a covering projection
+  scan (projected columns with a sort-key qual) answered as if the updated rows
+  had been removed.
+
+  The base table was always correct. The loss was confined to reads the planner
+  served from a projection, which makes it the worse shape: the same query
+  returns different answers depending on whether the projection is chosen.
+
+  `tuple_update` now calls the same fan-out that insert calls.
+
+  `docs/features.md`, `docs/administration.md` and `docs/how-to.md` said only
+  that inserts write projections. All three now say updates do too, and say why
+  a delete does not.
 - Arrow import reads the temporal unit and carrier width the file declares,
   rather than assuming the ones our own exporter writes (#864, #865).
 

docs/administration.md

@@ -248,7 +248,8 @@ SELECT pgcolumnar.add_projection(
 ```
 
 When you add the projection, pgColumnar fills it with the rows that exist. New inserts write to
-the base table and its projections. Projection scans are on by default
+the base table and its projections. Updates write there too, because an update
+creates a new row version. Projection scans are on by default
 (`pgcolumnar.enable_projection_scan`). Drop a projection with
 `pgcolumnar.drop_projection`.
 
@@ -261,7 +262,8 @@ it built. A second run builds nothing. A physical backup (`pg_basebackup`)
 preserves the projections themselves, which `test/replication.sh` verifies
 against a standby.
 
-A projection adds write cost and storage, because inserts write it too. Add one
+A projection adds write cost and storage, because inserts and updates both write
+it. Add one
 for a query pattern that a covering, sorted column subset serves, and measure the
 result. Confirm the plan uses it with `EXPLAIN`, which names the chosen projection.
 

docs/features.md

@@ -97,8 +97,10 @@ coverage.
   `pgcolumnar.add_projection(table, name, columns, sort_key)` declares an extra
   physical copy of a subset of the columns. That copy has its own sort order. It
   shares the row identity of the table. Every insert fans
-  out to each projection. A projection stored sorted has tight per-chunk minimum
-  and maximum ranges.
+  out to each projection. So does every update, because an update writes a new
+  row version with a new row number. A delete needs no fan-out, because the base
+  delete vector hides the old row number from projection scans. A projection
+  stored sorted has tight per-chunk minimum and maximum ranges.
 - The planner scans a projection instead of the base table when it covers the
   query's columns and its leading sort column is restricted. `EXPLAIN` shows
   `Columnar Projection: <name>`. Deletes and MVCC visibility come from the base,

docs/how-to.md

@@ -148,8 +148,8 @@ SELECT pgcolumnar.add_projection(
 EXPLAIN SELECT sum(amount) FROM events WHERE customer_id = 42;
 ```
 
-Adding a projection fills it from the existing rows. Later inserts write to the
-base table and to every projection, so each projection adds write cost. Drop one
+Adding a projection fills it from the existing rows. Later inserts and updates write to
+the base table and to every projection, so each projection adds write cost. Drop one
 with `pgcolumnar.drop_projection('events', 'events_by_customer')`.
 
 **Tuning.** Add a projection only for a hot access pattern the base sort order

src/columnar_tableam.c

@@ -1562,6 +1562,16 @@ pgcolumnar_tuple_update(COLUMNAR_TUPLE_UPDATE_ARGS)
 	rowNumber = PgColumnarWriteRow(writeState, rel, slot->tts_values,
 								 slot->tts_isnull);
 
+	/*
+	 * The new version is a new row number. Projections store that number as
+	 * their join key and skip deleted base numbers via the delete vector, so
+	 * DELETE needs no fan-out. UPDATE does: without it the projection still
+	 * holds the old number, the delete vector hides that number, and a covering
+	 * projection scan (or read_projection) answers as if the row is gone.
+	 */
+	PgColumnarProjectionFanoutRow(rel, writeState, rowNumber, slot->tts_values,
+								slot->tts_isnull);
+
 	/*
 	 * The new row version makes its block not all-visible, exactly as a plain
 	 * insert does (gap 28). This half of update-as-delete-plus-insert was the

src/columnar_write_state.c

@@ -2876,7 +2876,9 @@ pgcolumnar_flush_row_group(PgColumnarWriteState *writeState)
  * reusing the base stripe encoder (PgColumnarWriteRow + pgcolumnar_flush_row_group).
  * The base row number is stored as a leading int8 column so the projection can
  * be joined back to the base; deletes/visibility come from the base delete_vector, so
- * only INSERT fans out (see design/gaps/26-IMPL-projections-phase2-plan.md).
+ * DELETE does not rewrite the projection. UPDATE is delete-old plus insert-new
+ * and must fan the new number out or a covering projection scan will miss it
+ * (see design/gaps/26-IMPL-projections-phase2-plan.md).
  * ------------------------------------------------------------------------- */
 
 /* one buffered projection row: [rownumber, projcol1..projcolK] */

test/projection_update.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+#
+# pgColumnar: UPDATE must fan the new row number out to covering projections.
+#
+# A projection stores the base row number and filters with the base delete
+# vector, so DELETE needs no rewrite of the projection. UPDATE is delete-old
+# plus insert-new. The insert half used to skip fan-out, so the projection kept
+# only the deleted number. read_projection then returned no rows, and a planner
+# covering-projection scan (SELECT of projected columns with a sort-key qual)
+# answered as if the updated rows had been deleted.
+#
+# Usage:  test/projection_update.sh [PG_CONFIG]
+# Written fresh for pgColumnar.
+
+set -uo pipefail
+. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
+pgc_setup "${1:-/usr/local/pg17/bin/pg_config}"
+
+psql_run "CREATE TABLE pu (a int, b text, c int) USING pgcolumnar;"
+psql_run "SELECT pgcolumnar.add_projection('pu', 'pc', ARRAY['a','c'], ARRAY['c']);"
+psql_run "INSERT INTO pu SELECT g, 'r'||g, (g*7)%1000 FROM generate_series(1,20000) g;"
+psql_run "CREATE TABLE pu_h (a int, b text, c int) USING heap;"
+psql_run "INSERT INTO pu_h SELECT g, 'r'||g, (g*7)%1000 FROM generate_series(1,20000) g;"
+
+check "premise: planner uses the covering projection" \
+	"$(q "EXPLAIN (COSTS OFF) SELECT a, c FROM pu WHERE c BETWEEN 100 AND 200;" | grep -c 'Columnar Projection: pc')" "1"
+check "premise: projection matches the heap before any update" \
+	"$(pgc_set_hash "SELECT a, c FROM pu WHERE c BETWEEN 100 AND 200")" \
+	"$(pgc_set_hash "SELECT a, c FROM pu_h WHERE c BETWEEN 100 AND 200")"
+
+psql_run "UPDATE pu   SET a = a + 1 WHERE a % 10 = 0;"
+psql_run "UPDATE pu_h SET a = a + 1 WHERE a % 10 = 0;"
+
+check "covering projection scan matches heap after updating a projected column" \
+	"$(pgc_set_hash "SELECT a, c FROM pu WHERE c BETWEEN 100 AND 200")" \
+	"$(pgc_set_hash "SELECT a, c FROM pu_h WHERE c BETWEEN 100 AND 200")"
+check "read_projection matches the live base after that update" \
+	"$(pgc_set_hash "SELECT pgcolumnar.read_projection('pu','pc')")" \
+	"$(pgc_set_hash "SELECT a::text || '|' || c::text FROM pu")"
+check "full-table projection scan still matches heap" \
+	"$(pgc_set_hash "SELECT a, c FROM pu")" \
+	"$(pgc_set_hash "SELECT a, c FROM pu_h")"
+
+# A column the projection does not store still allocates a new base row number.
+psql_run "UPDATE pu   SET b = 'y' WHERE a % 7 = 0;"
+psql_run "UPDATE pu_h SET b = 'y' WHERE a % 7 = 0;"
+
+check "covering projection scan matches heap after updating a non-projected column" \
+	"$(pgc_set_hash "SELECT a, c FROM pu WHERE c BETWEEN 100 AND 200")" \
+	"$(pgc_set_hash "SELECT a, c FROM pu_h WHERE c BETWEEN 100 AND 200")"
+check "read_projection still matches the live base" \
+	"$(pgc_set_hash "SELECT pgcolumnar.read_projection('pu','pc')")" \
+	"$(pgc_set_hash "SELECT a::text || '|' || c::text FROM pu")"
+check "reconstruct via the projection still rebuilds every live row" \
+	"$(pgc_set_hash "SELECT pgcolumnar.reconstruct_via_projection('pu','pc')")" \
+	"$(pgc_set_hash "SELECT a::text || '|' || b || '|' || c::text FROM pu")"
+
+pgc_summary

test/run_all_versions.sh

@@ -238,6 +238,7 @@ SUITES=(
 	planner_choice_quality
 	preimage_rewrite
 	projection_privilege
+	projection_update
 	projections
 	pushdown_report
 	qual_order_selectivity