You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4cc2df7
Browse filesBrowse the repository at this point in the historyBrowse files
libsql-server: close out the low-severity review items
- `Reservation::publish` keeps the guard inside the reservation across
`install`'s awaits, so a cancellation there undoes the creation instead of
leaving the name pinned with the lock released.
- `remove_incomplete_marker` stats before taking the blocking-pool hop; every
cold load passes through it and the marker is almost always absent.
- Failing to remove the marker after publish is logged at `error` (it only
matters if the metastore is rebuilt from the filesystem meanwhile);
`FreshDir`'s log is `info` (client errors pass through it too); `acquire`
logs when it has to wait for an operation in progress.
- Restore the `Namespace` doc comment that had ended up on a helper.
- Tests: store-level `aborted_creation_is_undone` (`JoinHandle::abort`
mid-import; the waiting lookup sees "doesn't exist", the name is free);
the restart test drops the first server before starting the second; the
unmarked-directory test asserts the outcome instead of discarding it.
- Docs: 400 not "409-class"; the marker is acted upon in two places (create
and the startup sweep), both only for names the metastore doesn't know;
upgrade/downgrade notes; single-namespace-mode behaviour in ADMIN_API.md.
Copy file name to clipboardExpand all lines: docs/ATOMIC_NAMESPACE_CREATE_DESIGN.md
+8-4Lines changed: 8 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -201,7 +201,7 @@ Ok(ns)
201
201
-`pub(crate) const INCOMPLETE_MARKER: &str = ".incomplete";` in `namespace/mod.rs`.
202
202
- Created by `FreshDir::begin` for every brand-new directory.
203
203
- Removed by `Namespace::mark_complete(&self)` (`remove_file(self.path.join(MARKER))`, `NotFound` ok), called (a) by `create` after `flush`, (b) by `load_namespace`'s init after `make_namespace` — a namespace opened from the metastore is complete by definition, so lazily loaded namespaces (and single-namespace-mode auto-creation) never keep a marker.
204
-
-Consulted in exactly one place: `ConfigureNamespace::discard_incomplete(name)` (new trait method, default no-op; primary and schema share `helpers::discard_incomplete(base, name)` = `if dbs/<name>/.incomplete exists → remove_dir_all`). `create` calls it after `acquire`, i.e. only for a name with **no row and no loaded namespace** — the only situation in which a marked directory is unambiguously garbage. Lazy loads never see it, so a stale marker on a published namespace (crash between `flush` and `mark_complete`) is removed, never acted upon.
204
+
-Acted upon only for a name the metastore does **not** know: by `ConfigureNamespace::discard_incomplete(name)` (new trait method, default no-op; primary and schema delegate to `incomplete::discard_if_incomplete`), which `create` calls after `acquire` — i.e. with no row and nothing loaded — and by the startup `sweep` under the same condition. That is the only situation in which a marked directory is unambiguously garbage. Lazy loads never act on it, so a stale marker on a published namespace (crash between `flush` and unmark) is removed, never acted upon. The metastore's filesystem recovery (`maybe_recover_from_fs`, empty metastore only) skips marked directories rather than adopting them.
205
205
-`MetaStoreInner::maybe_recover_from_fs` (metastore empty ⇒ adopt `dbs/*`) skips directories carrying the marker, so a crash during the very first creation on a fresh server does not get adopted as a namespace at the next start.
206
206
207
207
A directory **without** a marker and without a row keeps today's semantics (`Dump` ⇒ `LoadDumpExistingDb`; `Latest` ⇒ adopted). Such directories can only come from pre-marker servers or manual intervention; being destructive there is not worth it.
@@ -224,7 +224,7 @@ Replace the `Bomb` + manual `to_lock` with `Reservation::acquire(to)`; order bec
224
224
## 5. Flow for a dump import (streaming or buffered)
225
225
226
226
1. Admin handler resolves `dump_url` (errors here touch nothing).
227
-
2.`create` → `Reservation::acquire` (409-class error if the name is loaded or has a row).
227
+
2.`create` → `Reservation::acquire` (`NamespaceAlreadyExist`, HTTP 400, if the name is loaded or has a row).
228
228
3. In-memory config; `discard_incomplete` (removes a crashed previous attempt, if any).
@@ -243,7 +243,7 @@ Replace the `Bomb` + manual `to_lock` with `Reservation::acquire(to)`; order bec
243
243
| between `publish` and marker removal | — (published; a stale marker is harmless) | same | row + marker + complete data ⇒ serves; marker removed on first lazy load |
244
244
| after `publish`| — | response may be lost ⇒ client retries ⇒ `NamespaceAlreadyExist` ⇒ by the contract, complete | same |
245
245
246
-
Client rule: **2xx ⇒ complete; anything else ⇒ retry; `NamespaceAlreadyExist` on retry ⇒ complete.** No inspection of tables needed.
246
+
Client rule: **2xx ⇒ complete; anything else ⇒ retry; `NamespaceAlreadyExist` on retry ⇒ complete.** No inspection of tables needed. (For namespaces *created* through this path. The single-namespace-mode config upsert of the default namespace, and ghost rows left by older binaries, are outside it.)
247
247
248
248
## 7. Concurrency
249
249
@@ -289,7 +289,11 @@ Reviewers (correctness, architecture, testing, operations) plus a manual pass fo
289
289
| MED | Cancel tests could fire before the creation started (turmoil latency) and so pass vacuously; the restart test never reached the `\|\| exists` guard. | Cancel once `dbs/foo/.incomplete` exists; create before any lookup after restart; "usable afterwards" assertion restored. |
290
290
| LOW | Marked orphans were only reclaimed by a `create` of the same name; `maybe_recover_from_fs` skip untested; success not logged. | Startup `sweep`; `recovery_from_fs_skips_marked_directories`, `sweep_cleans_leftovers_of_a_crash`; `info!("namespace created")` with elapsed, `warn!` with the error on failure, distinct message on cancellation. |
291
291
292
-
Not changed: bottomless objects uploaded under a never-published db_id are inert junk (as today); `fsync` of the marker (power loss between creating and syncing it could leave an unmarked orphan — accepted); the moka `get_with(default)` leftover in `checkpoint` is healed lazily rather than fixed at the source.
292
+
LOW items closed in the follow-up commit: `publish` keeps the guard inside the reservation across `install`'s awaits, so a cancellation there undoes the creation instead of leaking the pin; `remove_incomplete_marker` stats before taking the blocking-pool hop (every cold load passes through it); a failure to remove the marker after publish is logged at `error`; `FreshDir`'s log is `info` (every failed create, including client errors, passes through it); `acquire` logs when it has to wait; store-level `aborted_creation_is_undone` test (`JoinHandle::abort` mid-import); the restart test drops the first server before starting the second; the unmarked-directory test asserts the outcome.
293
+
294
+
Not changed: bottomless objects uploaded under a never-published db_id are inert junk (as today); `fsync` of the marker (power loss between creating and syncing it could leave an unmarked orphan — accepted); the moka `get_with(default)` leftover in `checkpoint` is healed lazily rather than fixed at the source; after an *abort* the (empty) cache entry stays until a lookup heals it or the cache evicts it — it counts toward capacity but is never served; a metastore rebuilt from the filesystem (`destroy_on_error` + `allow_recover_from_fs`) in the milliseconds between a namespace's flush and its unmark would skip that namespace.
295
+
296
+
**Upgrade / downgrade.** Ghost rows and unmarked orphan directories left by failed creates on *older* binaries are untouched (everything here is marker-gated); for them the old remedy stays: `DELETE /v1/namespaces/:ns`, then create again. Downgrading the binary is safe: the metastore schema and the config protobuf are unchanged; an old binary ignores `.incomplete` and `.discarded/` (it may adopt a marked orphan via `maybe_recover_from_fs` on an empty metastore, as it always could with any leftover directory).
0 commit comments