Skip to content

Replace pickers with composable Store backends - #273

Open
bruno-f-cruz wants to merge 11 commits into
mainfrom
feat-stores
Open

Replace pickers with composable Store backends#273
bruno-f-cruz wants to merge 11 commits into
mainfrom
feat-stores

Conversation

@bruno-f-cruz

@bruno-f-cruz bruno-f-cruz commented Sep 8, 2026

Copy link
Copy Markdown
Member

Retires clabe.pickers in favour of a composable Store — a four-method interface over the databases that hold rigs, task logic, trainer state and per-animal reconfigurations.

Implements the Pickers to Stores plan. Clean break — no deprecation shims. A companion PR against Aind.Experiment.VrForaging follows.

Why

DefaultBehaviorPicker fused data access, user prompting, Session construction and recency caching into one object, and DataversePicker subclassed it purely to swap one record — trainer state — for a different backend. That inheritance is the defect; composition was what was wanted.

What lands

clabe.ui — ambient frontend. Module-level passthroughs for every Frontend method, finishing the pattern notify() already started. Output methods (notify, header, activity) no-op headless; prompts raise NoFrontendError rather than hanging on a rig. Adds use_frontend() as a context-manager form of set_current_frontend for tests.

clabe.stores

Piece Role
Kind[T] Binds a record name to its pydantic model. from_rig / from_task / from_session / from_trainer_state fix the canonical names; the bare constructor snake-cases the model name for custom records.
Store resolve (interactive) · list (headless) · write · scoped
StoreBase Scope narrowing plus the default resolve policy: none raises, one auto-selects, many prompt.
CompositeStore Routes per kind name to a backend.
LocalFileStore The config library, behind a swappable Layout.
DataverseStore Trainer state only, off the picker inheritance. Lives at clabe.stores.dataverse and is not re-exported from clabe.stores, because msal is a Windows-only dependency of the aind-services extra — the same convention as data_transfer.aind_watchdog.
MemoryStore In-memory fake, for tests and headless assembly.
from clabe.stores import CompositeStore, LocalFileStore
from clabe.stores.dataverse import DataverseStore

store = CompositeStore(
    default=LocalFileStore(root=VR_LIB),
    routes={"trainer_state": DataverseStore()},
)
store = store.scoped(subject=session.subject)

SessionBuilder (clabe/session.py) keeps the one genuinely interactive job with no database behind it. It holds no hidden state, so recovery flows narrow a store explicitly instead of poking picker._session.

ByAnimalModifier moves to clabe.modifiers and reads/writes through a Store, so it stops reaching into picker.subject_dir.

Call-site accounting

Today Becomes
picker.pick_rig(Model) store.resolve(VR_RIG)
picker.pick_task(Model) store.resolve(VR_TASK)
picker.pick_trainer_state(Model) store.resolve(SUGGESTION)
picker.push_new_suggestion(ts) store.write(SUGGESTION, ts)
picker.dump_model(rig) store.write(VR_RIG, rig)
picker.pick_session(Session) SessionBuilder(launcher).build()
picker.subject_dir store.scoped(subject=...)

Hard constraint honoured

DefaultLayout reproduces the existing tree on \allen\aind\scratch\AindBehavior.db\... byte-for-byte: Rig/<COMPUTERNAME>/*.json, Task/*.json, Subjects/<subject>/{task,trainer_state}.json. No migration. The library-wide vs subject-specific task lookup — a nested try/while in pick_task — is now an ordered scope chain: subject folder first, then the shared library.

Deviations from the plan, and why

  • ConfiergeStore is not in this PR. ConfiergeService only exists on the unmerged feat-add-ficus-adapter branch. The adapter slots in with no base-class change once that lands.
  • SessionBuilder.subject_source dropped. The plan marked it an extension point with no caller; there is no sensible Kind for a bare subject string. Add it when something needs it.
  • Bare-model automagic adopted. store.resolve(ManipulatorPosition) wraps the model in Kind(...) internally. The naming trap (Kind(AindVrForagingRig)"aind_vr_foraging_rig", not "rig") is documented on Kind.
  • Recency cache is now list ordering, not a separate prompt. The picker asked "pick from cache / select from library" as two steps; resolve now floats recently-chosen options to the top of one list. The picker's use_cache flag is gone — ordering never changes what is selectable.
  • Trainer-state checks moved onto Kind.from_trainer_state's validator (reject stageless, warn off-curriculum), so both backends get them from one place.
  • Overwrite-confirmation prompt on write is gone, per the plan: clobber behaviour is the store's contract.

Open

  • write returns None. Revisit if a caller needs the written id.
  • LocalFileStore.resolve offers the pick list only; no file-browse escape hatch yet.

Retires clabe.pickers in favour of a Store protocol (resolve/list/write/scoped)
with a Kind descriptor binding a record name to its pydantic model.

- clabe.ui gains ambient passthroughs for every Frontend method, so stores and
  builders no longer thread a frontend. Output methods no-op headless; prompts
  raise NoFrontendError.
- clabe.stores: Kind, Store, StoreBase, CompositeStore, LocalFileStore (with a
  Layout reproducing today's config-library tree), DataverseStore, MemoryStore.
- SessionBuilder keeps the one interactive job with no database behind it, and
  drops the makedirs side effect from session construction.
- ByAnimalModifier moves to clabe.modifiers and reads/writes through a Store.
MemoryStore appends rather than replacing, as Dataverse does: a fake that can
only ever hold one record per kind and scope cannot stand in for a store with
several candidates, and rebinding its record list broke scoped views sharing
the same data.
Examples declare their record kinds once in _mocks.py and use the ambient
prompts, which is the pattern the refactor is for.

The mkdocs nav is the output of docs/build.py, so it also picks up sections
that had drifted out of the committed file (web, aind_apps, logging, runnable,
aind_validators). CI regenerates it before deploying either way.
bruno-f-cruz and others added 8 commits September 8, 2026 08:56
msal is a Windows-only dependency of that extra, so eagerly importing the
Dataverse backend from stores/__init__.py broke every import of clabe.stores on
Linux. Dataverse becomes a public submodule imported explicitly, matching how
data_transfer exports robocopy but not aind_watchdog.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant