[adsharma]

Concurrent write transactions are production-ready, so the debug_ prefix has to go.

What changed

• Setting renamed: debug_enable_multi_writes -> enable_multi_writes (consistent with SystemConfig::enableMultiWrites and the other enable_* settings).
• C++ struct renamed: EnableMVCCSetting -> EnableMultiWritesSetting; dropped the stale TODO calling it a temporary test solution.
• Default flipped from false to true in SystemConfig.
• Updated all CALL sites (e2e .test files, transaction_test.cpp, checkpoint_test.cpp, node_write_bench.cpp).
• ddl_serial_execution.test MultiWritesException case now opts out explicitly with CALL enable_multi_writes=false so single-writer mode stays covered; the EnableMultiWrites case now exercises the false->true toggle.

Verified locally

• ninja -C build/release lbug builds clean.
• e2e concurrency suites pass (27 tests): ddl_serial, ddl_concurrent, dml_empty_serial, dml_tinysnb_serial.

Companion binding PRs

Each binding now lets the engine control defaults (unset = engine default), so no binding pins a copy of these defaults anymore:

• Python: feat(config): let the engine control SystemConfig defaults ladybug-python#77
• Java: feat(config): let the engine control SystemConfig defaults ladybug-java#18
• Rust: feat(config): let the engine control SystemConfig defaults ladybug-rust#45
• Node.js: feat(config): let the engine control SystemConfig defaults ladybug-nodejs#28
• WASM: feat(config): let the engine control SystemConfig defaults ladybug-wasm#14

[adsharma]

Note: this is 0.23.x material — it flips a user-visible default (multi-writer mode on by default) and renames a setting, so it should ride the 0.23 release, not get backported to a 0.22.x patch.

[adsharma]

Companion binding PRs (same default flip, separate repos since these are submodules here): ladybug-python#76, ladybug-java#17, ladybug-rust#44. nodejs/wasm don't expose the flag, so no change needed there. CI on this PR covers the engine side.

[adsharma]

Update: superseded the default-flip-only binding PRs with fuller ones where each binding lets the engine control all defaults (unset = engine default): ladybug-python#77, ladybug-java#18, ladybug-rust#45, plus nodejs#28 and wasm#14 which needed the same treatment. CI here covers the engine side; each binding repo's CI covers its own.

[zahariash]

I checked this against the open PRs under #1376, simulating the new default on their merge.

Automatic checkpoints wait for other writers. A commit's automatic checkpoint takes the write gate and waits up to 5 s for every other write transaction to leave. In single-writer mode none can be open at that point. Here, one connection held a write transaction open while another committed three single-row writes, with checkpoint_threshold=1:

	Each write	Caller sees
0.22.0, single-writer	0 s	"Only one write transaction at a time" (nothing applied)
0.22.0, multi-writer	5.6 s	"Transaction committed successfully, but the post-commit checkpoint failed" (#1199)
#1376 merge, multi-writer	5.6 s	success, and a stderr warning per commit

With the 16 MB default this starts once the WAL passes the threshold and repeats on every commit while a long write transaction stays open, and new writers wait behind the gate meanwhile. So #1361 should land with or before this PR; otherwise retries of those "failed" commits apply them twice. The 5 s stall stays even with #1361; an automatic checkpoint could skip, instead of waiting, while other write transactions are open.

#1375 still matters. In multi-writer mode a BEGIN is still refused when the WAL is poisoned. With one failed WAL sync injected, 0.22.0 refuses the BEGIN and the connection's next MATCH segfaults (#1316); with #1375 it answers.

Merge notes.

• fix(txn): keep a connection usable after a refused BEGIN #1375's tests rely on the single-writer refusal: RefusedBeginKeepsAutoMode and RefusedPreparedAndMultiStatementBeginKeepAutoMode fail with the new default, and RefusedBeginKeepsConnectionUsable in ddl_serial_execution.test conflicts with this PR's edit there. Whichever lands second needs CALL enable_multi_writes=false in them.
• fix(txn): don't fail a committed write when its automatic checkpoint fails #1361's AutoCheckpointBlockedByOpenWriterDoesNotFailCommit calls debug_enable_multi_writes, which this PR renames without an alias.

Apart from those, transaction_test and the whole e2e suite pass on the #1376 merge with multi-writer mode as the default. I skipped EmptyDBTransactionTest, whose concurrency tests already turn the mode on themselves.

The simulator and the retention loop behind #1436 and #1437 have only run single-writer mode so far; the C stress loop ran multi-writer mode in half of its rounds without a violation.

[adsharma]

@zahariash what are the remaining blockers?

[zahariash]

Four, checked today on main 1a45f1b with this PR merged in, runtime checks on:

1. A row inserted while another transaction's ALTER TABLE ADD commits gets 0 in the new column.

   conn1: CREATE NODE TABLE T(id INT64 PRIMARY KEY, b INT64);
   conn1: CREATE (:T {id: 1, b: 10});
   conn2: BEGIN TRANSACTION;
   conn2: ALTER TABLE T ADD x INT64 DEFAULT 1;
   conn1: BEGIN TRANSACTION;
   conn1: CREATE (:T {id: 2, b: 20});
   conn2: COMMIT;
   conn1: COMMIT;
   conn1: MATCH (t:T) RETURN t.* ORDER BY t.id;   -- 1|10|1, 2|20|0
   conn1: CHECKPOINT;   -- fails numResidentRows == ...getNumValues() (node_group.cpp:901)
   

   id 2 gets x = 0, neither its default nor NULL. The ALTER adds x to the table's storage right away, while conn1 commits with the columns of its snapshot, so x ends up one value short. A release build of main 31f728b passes the CHECKPOINT and keeps the 0. A read in conn1's transaction after conn2's COMMIT fails columnIDs.size() == other.columnStats.size() (table_stats.h) instead and aborts it, and that read fails the same way when the ADD rolls back. #1470 rewrites that stats merge, but on its 10-30 head the row still gets 0 and the CHECKPOINT assertion stays. 2bb9fa7b4's conflict check doesn't apply, since the insert writes no catalog entry. It needs multi-writer mode. I'll file it.

2. COPY FROM while another write transaction is open reports a committed copy as failed. Its forced checkpoint waits 5 s for the other writer, then the statement fails with "Transaction committed successfully, but the checkpoint it requires failed. Its changes are not in the WAL and may be lost on restart unless a later CHECKPOINT succeeds". The copied rows are visible, and a retry fails on a duplicate primary key. Without the other writer the same COPY succeeds, and single-writer mode refuses its transaction up front. CREATE INDEX on a large ART takes the same forced-checkpoint path. Skipping the wait can't help here, since that checkpoint is what makes the data durable.

3. Past the checkpoint threshold, each commit waits up to 5 s while another write transaction is open. Still open from my 10-27 comment, now with #1361's stderr warning instead of an error: three commits took 16.9 s against 0.3 s. Single-writer mode refuses the second writer instead. By the code, a write BEGIN that arrives during the wait holds the lock every BEGIN, COMMIT and ROLLBACK takes, read-only ones included (transaction_manager.cpp L70-L75), so from then on the open writer can't commit either and every connection waits out the 5 s. I'd make the automatic checkpoint skip while other writers are open: take the gate, and if one is still active, release it before any storage phase and leave the checkpoint to a later commit. Past a WAL cap it would fall back to today's wait. I can send that as a PR; it doesn't cover the forced checkpoints in 2.

4. Seven tests fail. This PR's CI on 10-30 failed the three from #1375 that rely on the single-writer refusal: RefusedBeginKeepsAutoMode, RefusedPreparedAndMultiStatementBeginKeepAutoMode and RefusedBeginKeepsConnectionUsable. Four tests that landed after that run call debug_enable_multi_writes, which this PR removes without an alias: #1361's AutoCheckpointBlockedByOpenWriterDoesNotFailCommit and the three write-write conflict cases from 2bb9fa7b4. With the patch below the full suite passes (3314/3314).

Patch for 4, for this PR merged with current main

diff --git a/test/test_files/transaction/concurrency/ddl_serial_execution.test b/test/test_files/transaction/concurrency/ddl_serial_execution.test
index 87ddf4e17..22f13e96b 100644
--- a/test/test_files/transaction/concurrency/ddl_serial_execution.test
+++ b/test/test_files/transaction/concurrency/ddl_serial_execution.test
@@ -20,6 +20,8 @@ Cannot start a new write transaction in the system. Only one write transaction a
 1|b|INT32|NULL|False
 
 -CASE RefusedBeginKeepsConnectionUsable
+-STATEMENT CALL enable_multi_writes=false;
+---- ok
 -STATEMENT CREATE NODE TABLE t(a INT, PRIMARY KEY(a));
 ---- ok
 -STATEMENT BEGIN TRANSACTION;
@@ -310,7 +312,7 @@ Catalog exception: t does not exist in catalog.
 1|c|INT32|NULL|False
 
 -CASE AlterTableAddWriteWriteConflictUncommitted
--STATEMENT CALL debug_enable_multi_writes=true;
+-STATEMENT CALL enable_multi_writes=true;
 ---- ok
 -STATEMENT CREATE NODE TABLE t(a INT, b INT, PRIMARY KEY(a));
 ---- ok
@@ -334,7 +336,7 @@ Catalog exception: Write-write conflict on altering catalog entry with name t.
 2|x|INT32|NULL|False
 
 -CASE AlterTableAddWriteWriteConflictCommitted
--STATEMENT CALL debug_enable_multi_writes=true;
+-STATEMENT CALL enable_multi_writes=true;
 ---- ok
 -STATEMENT CREATE NODE TABLE t(a INT, b INT, PRIMARY KEY(a));
 ---- ok
@@ -358,7 +360,7 @@ Catalog exception: Write-write conflict on altering catalog entry with name t.
 2|x|INT32|NULL|False
 
 -CASE AlterVsDropWriteWriteConflict
--STATEMENT CALL debug_enable_multi_writes=true;
+-STATEMENT CALL enable_multi_writes=true;
 ---- ok
 -STATEMENT CREATE NODE TABLE t(a INT, b INT, PRIMARY KEY(a));
 ---- ok
diff --git a/test/transaction/checkpoint_test.cpp b/test/transaction/checkpoint_test.cpp
index e720ba1be..a22419faa 100644
--- a/test/transaction/checkpoint_test.cpp
+++ b/test/transaction/checkpoint_test.cpp
@@ -928,7 +928,7 @@ TEST_F(PostCommitCheckpointFailureTest, AutoCheckpointBlockedByOpenWriterDoesNot
     createDB(10000 /* checkpoint wait timeout in microseconds */);
     conn = std::make_unique<main::Connection>(database.get());
     checkpointAfterEveryCommit();
-    ASSERT_TRUE(conn->query("CALL debug_enable_multi_writes=true;")->isSuccess());
+    ASSERT_TRUE(conn->query("CALL enable_multi_writes=true;")->isSuccess());
     auto context = getClientContext(*conn);
     main::Connection other{database.get()};
     ASSERT_TRUE(other.query("BEGIN TRANSACTION;")->isSuccess());
diff --git a/test/transaction/transaction_test.cpp b/test/transaction/transaction_test.cpp
index bbaba407a..b66f1cc6d 100644
--- a/test/transaction/transaction_test.cpp
+++ b/test/transaction/transaction_test.cpp
@@ -48,6 +48,7 @@ TEST_F(PrivateApiTest, TransactionModes) {
 }
 
 TEST_F(PrivateApiTest, RefusedBeginKeepsAutoMode) {
+    ASSERT_TRUE(conn->query("CALL enable_multi_writes=false;")->isSuccess());
     auto conn2 = std::make_unique<lbug::main::Connection>(database.get());
     ASSERT_TRUE(conn->query("BEGIN TRANSACTION;")->isSuccess());
     ASSERT_FALSE(conn2->query("BEGIN TRANSACTION;")->isSuccess());
@@ -60,6 +61,7 @@ TEST_F(PrivateApiTest, RefusedBeginKeepsAutoMode) {
 }
 
 TEST_F(PrivateApiTest, RefusedPreparedAndMultiStatementBeginKeepAutoMode) {
+    ASSERT_TRUE(conn->query("CALL enable_multi_writes=false;")->isSuccess());
     auto conn2 = std::make_unique<lbug::main::Connection>(database.get());
     ASSERT_TRUE(conn->query("BEGIN TRANSACTION;")->isSuccess());
     auto prepared = conn2->prepare("BEGIN TRANSACTION;");

[adsharma]

I've fixed as many of the issues as possible. Let's deal with the rest in separate PRs.

@aikins01:

• checkWWConflict is handling DDL-vs-DDL races
• schemaEpoch is handling DDL-vs-DML races

[aikins01]

Thanks, I'll take the rest in follow-up PRs. Running the merged fix, I found these too:

• An insert into a renamed table can fail with the new conflict error, with nothing else running: ALTER TABLE A RENAME TO B, create ~20 more tables, then CREATE (:B {...}). getHeadEntryOfOID can return the renamed-from tombstone, which shares the OID.
• BEGIN; CREATE (:P {...}); ALTER TABLE P DROP a; COMMIT; fails the same way. dropColumn() bumps the epoch but doesn't refresh the transaction's own local table like addColumn does.
• The check runs in publishCommit(), after writeCommitToWAL(), so a rejected commit is still in the WAL. Reopening without a checkpoint brings back 2|20|0 for the repro from the 11-02 comment.
• A COPY run while another write transaction is open now returns success, and its rows are gone after a restart unless something checkpoints first. The warning only goes to stderr.

I'll fix the first three together, since they're the same check. For COPY, should it fail like before, saying the commit happened and shouldn't be retried, or wait until it can checkpoint?

[aikins01]

#1506 covers the first three issues from my last comment. The insert followed by DROP in one transaction no longer reports a conflict there, but its COMMIT still runs into #1221.

On the COPY question, I went with waiting in #1507. A COPY, or a CREATE INDEX on a large ART, now waits for the other write transactions to finish before it commits, and new write transactions wait at BEGIN meanwhile, so its checkpoint can run. If another writer is still open after the checkpoint wait timeout, the COPY is rolled back with nothing committed, so running it again is safe. A forced checkpoint that fails after the commit for another reason throws again instead of warning.

The cost is that a long-running write transaction can now make a COPY fail. Does that work for you, or would you rather it fail after committing like before?

[adsharma]

https://github.com/LadybugDB/ladybug/actions/runs/41811928021/job/126513303014 test failure seems related.