feat(agents): generic externally-writable custom collections (comments as first consumer)

[balegas]

Summary

An alternative to #4529 that delivers the same session-comments feature, but built on a generic, extensible interface instead of hardcoding comments into the entity schema. Comments becomes one custom collection among potentially many; the mechanism underneath is reusable.

Three capabilities the agent runtime didn't have before:

1. Opt-in externally-writable collections. Entity state is agent-owned. A collection becomes writable from the HTTP router only when its definition sets externallyWritable — everything else stays agent-only by default. (Every collection is always writable by the agent; only an opt-in subset is writable externally, hence the name.)
2. Authenticated, auditable writes. Router writes require authentication. The server stamps the authenticated principal into the change-event header (provenance, outside the user payload), and the client materializes it into a read-only virtual column (_principal). A client can't spoof it via value.
3. Schema-validated writes. Each collection carries a schema; router writes validate value server-side via the existing validateWriteEvent before append.

Design

Runtime (@electric-ax/agents-runtime)

• CollectionDefinition.externallyWritable?: boolean.
• At registration, writable collections are emitted as externally_writable_collections ({ type } per collection).
• The entity stream DB materializes headers.principal into the fixed _principal virtual column, mirroring _timeline_order, and strips it (with _seq/_timeline_order) before client write-back.
• createEntityTimelineQuery accepts customSources — consumer-provided sources unioned into the timeline query. The runtime itself knows nothing about specific custom collections, and the agent's LLM context never includes them.

Server (@electric-ax/agents-server)

• externally_writable_collections persisted as a jsonb column on entity_types (migration 0016) and resolved via getEffectiveSchemas. Legacy principalColumn in registration payloads is accepted and ignored for version-skew tolerance.
• EntityManager.writeCollection gates on it (403 if not writable), enforces entity-status/fork-lock, stamps headers.principal = { url, kind, id }, validates value, and appends.
• POST /:type/:instanceId/collections/:collection exposes it (auth + write permission, same middleware chain as send). Single POST, operation in the body, 201 insert / 200 update·delete.

Comments consumer (@electric-ax/agents + @electric-ax/agents-server-ui)

• commentsCollection (schema ported from feat(agents): Add session comments to agent timelines #4529, minus from_principal) declared as state on Horton and worker.
• The UI projects comments into the timeline by passing a comments customSources entry to the timeline query (author resolved from _principal); comments are a UI-level concern, hardcoded only in agents-server-ui.
• The feat(agents): Add session comments to agent timelines #4529 comments UI is cloned and re-sourced: writes via an optimistic TanStack action backed by the authenticated /collections/comments endpoint; reads comment.from.

Test Plan

• @electric-ax/agents-runtime — typecheck + 839 tests (incl. principal virtual-column materialization, write-back stripping, registration emit, timeline projection)
• @electric-ax/agents-server — typecheck + suite (incl. writeCollection 403/409 gating, principal-header stamping, schema validation, externally_writable_collections jsonb round-trip)
• @electric-ax/agents — typecheck + 60 tests (comments declared on Horton/worker)
• @electric-ax/agents-server-ui — typecheck + 88 tests (optimistic comment write to /collections/comments, author from _principal)
• Manual: post a comment from the UI, confirm it syncs, right-aligns for the author, and the principal is recorded in the change-event header

🤖 Generated with Claude Code

[github-actions]

Electric Agents Desktop Builds

Build artifacts for commit 31db3c0.

Platform	Status	Artifact
macOS Apple Silicon	Passed	DMG
macOS Intel	Passed	DMG
Windows x64	Passed	Installer
Linux x64	Passed	AppImage / deb

Workflow run

[github-actions]

Electric Agents Mobile Build

Local mobile checks ran for commit 31db3c0.

The EAS Android preview build was skipped because the mobile-eas-build label is not present.
Add the mobile-eas-build label to this PR to produce an installable preview build.

Workflow run

[codecov]

Codecov Report

❌ Patch coverage is 45.40881% with 434 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.53%. Comparing base (1219cb4) to head (31db3c0).
⚠️ Report is 8 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...agents-server-ui/src/components/EntityTimeline.tsx	0.00%	140 Missing ⚠️
...s/agents-server-ui/src/components/MessageInput.tsx	0.00%	75 Missing ⚠️
...agents-server-ui/src/components/views/ChatView.tsx	0.00%	49 Missing ⚠️
.../agents-server-ui/src/components/CommentBubble.tsx	0.00%	34 Missing ⚠️
...ages/agents-server-ui/src/hooks/useForkFromHere.ts	0.00%	25 Missing ⚠️
packages/agents-server-ui/src/lib/principals.ts	0.00%	22 Missing ⚠️
...s-server-ui/src/components/workspace/SplitMenu.tsx	0.00%	13 Missing ⚠️
packages/agents-runtime/src/create-handler.ts	87.91%	11 Missing ⚠️
packages/agents-runtime/src/entity-stream-db.ts	85.48%	9 Missing ⚠️
.../agents-server-ui/src/components/AgentResponse.tsx	0.00%	8 Missing ⚠️
... and 12 more

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #4551       +/-   ##
===========================================
- Coverage   70.96%   58.53%   -12.44%     
===========================================
  Files          83      375      +292     
  Lines        9856    41246    +31390     
  Branches     3124    11826     +8702     
===========================================
+ Hits         6994    24142    +17148     
- Misses       2844    17028    +14184     
- Partials       18       76       +58     

Flag	Coverage Δ
packages/agents	71.37% <ø> (ø)
packages/agents-mcp	77.70% <ø> (?)
packages/agents-mobile	75.49% <ø> (?)
packages/agents-runtime	83.06% <89.65%> (?)
packages/agents-server	74.94% <85.48%> (+0.10%)	⬆️
packages/agents-server-ui	7.42% <19.96%> (?)
packages/electric-ax	46.42% <ø> (ø)
packages/experimental	87.73% <ø> (?)
packages/react-hooks	86.48% <ø> (?)
packages/start	82.83% <ø> (?)
packages/typescript-client	91.83% <ø> (?)
packages/y-electric	56.05% <ø> (?)
typescript	58.53% <45.40%> (-12.44%)	⬇️
unit-tests	58.53% <45.40%> (-12.44%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[msfstef]

We generally do patch changes still even for breaking changes

[balegas]

Done — switched all four entries to patch in 9673112.

[msfstef]

Design question: is comments meant to be per-agent or platform-level?

The externally-writable safeguard itself makes sense to me — default-deny, agent owns its state. But the PR seems ambiguous about whether the comments collection's existence is meant to be optional:

• Write side: comments only exists because horton/worker explicitly declare state: { comments: commentsCollection }. Any agent definition that omits that line silently loses commenting.
• Read side: the UI assumes comments everywhere — UI_ENTITY_CUSTOM_STATE registers it on every entity stream, the Comment tab and comments-only view are offered for all chat entities, and posting to a type that didn't declare comments fails only at submit time with a 403.

The design spec (dropped in 13050d0) justifies the writability opt-in, but I couldn't find a rationale for making existence opt-in per agent — it seems to just fall out of the mechanism.

If comments is a platform feature, could it join the default layer instead — e.g. merged into every registration alongside DEFAULT_STATE_SCHEMAS, with its externally_writable_collections entry — so the server-side truth matches what the UI already assumes? If it's genuinely per-agent, the UI should gate the comment affordances on the type's externally_writable_collections (already synced to the client). As-is we have per-agent boilerplate on the write side and an unconditional promise on the read side.

[msfstef]

Provenance is "last touch", which the comments feature can't actually be built on

The header-principal design answers who performed this write — but comments need who authored this row, and for any operation other than insert those are different questions:

• There's no stored author field (from_principal was deliberately dropped); authorship exists only as _principal, materialized from the latest event's header.
• Updates are full row replaces, and the router stamps the caller's principal on every event. So B updating A's comment replaces the content and flips the displayed author to B. A's authorship survives only in raw stream history, which nothing reads.
• Agent-side updates are worse: no principal header + full replace means the row comes out with no _principal at all.
• The schema's edited_at/deleted_by fields imply edits/soft-deletes are intended — but deleted_by is client-supplied in value (spoofable), and the provenance model can't represent "created by A, edited by B" at all.

Combined with no ownership checks (the dropped spec records this as a decision: any principal with entity write may perform any operation), anyone who can prompt the session can delete or rewrite anyone's comment — and rewriting silently reattributes it.

Insert-only is the case where the provenance model is actually sufficient, so concretely: could ExternallyWritableCollectionConfig grow an operations allowlist (e.g. { type, operations: ['insert'] }), with comments shipping insert-only? That closes the moderation/reattribution hole without the server needing to materialize state for row-level ownership checks, and update/delete can be enabled later once there's a durable-authorship story (server-echoed creator, ownership-gated ops, or an explicit author field).

[balegas]

Design question: is comments meant to be per-agent or platform-level?

We decided that comments would only be a feature for horton and hard-code it to the ui. I think this is limiting, so I introduced contracts and entities can opt-in for them. The ui will render comments for any entity with that capability.

• The canonical commentsCollection declares a contract: 'comments/v1' marker, forwarded in type registration.
• The server reserves the comments collection name for that contract (and vice versa), so an agent's unrelated writable collection can never be mistaken for platform comments.
• The entity GET response now includes the type's externally_writable_collections, and the UI registers the comments collection on the entity stream only when the contract is advertised — the Comments view, composer Comment tab, timeline merge, and display toggle are all gated on it, so the submit-time 403 is no longer reachable through the UI.

[balegas]

Provenance is "last touch", which the comments feature can't actually be built on

Provenance here is intentionally attributed to the mutation: _principal answers "who performed the write that produced the current value", which is the correct attribution for materializing the latest value of a row — it isn't trying to be a durable authorship record. You're right that durable authorship ("created by A, edited by B") isn't representable in the materialized row today, and that anyone with entity write can rewrite or remove a comment — we're consciously accepting that for now: comments inherit entity-level write semantics.

The fuller treatment, if/when we want it, is additive on top of this design rather than a rework of it:

• Comment history — every revision (with its principal header) is already durably retained in the stream; surfacing it is a read-side feature, no write-path changes needed.
• Permissions on the state contract — an operations allowlist like the one you sketched (or ownership rules) hangs naturally off the collection config / the comments/v1 contract added for the per-agent gating.

So: deferring both and leaving the scope of this PR as is. Happy to open a follow-up issue capturing the moderation/authorship gaps so they don't get lost.