Add inspectable resumable manifests for bulk operations #60
Description
Product decision
Operators need a supported inspectable bulk cancellation/signaling operation that survives client interruption. The current CLI collects results in memory; final JSON output is not a durable resume journal. Replace durable-workflow/durable-workflow.github.io#273's suggestion that every operator build this independently with a supported CLI workflow.
Current CancelCommand is bounded but process-local; SignalCommand targets one workflow/run.
Acceptance
- Persist a bounded selection manifest before mutation, containing namespace, exact workflow/run IDs, operation parameters and stable per-target operation identities. Explain that paginated visibility queries are not transactional snapshots.
- Persist per-target outcomes incrementally and expose progress, inspection and explicit resume. A restart must not silently relist a broader selection or retarget a successor run.
- Distinguish acknowledged completion, permanent refusal, retryable pressure, skipped/closed target and ambiguous admission. Confirmed actions are not submitted again merely because the client restarted.
- Signal/update retries use the durable request identity contract in Make signal and update retries safe after a lost response .github#216, or remain explicitly unresolved until that contract is available. Never equate deduplicated command admission with exactly-once external effects.
- Bound concurrency, rate, manifest size and retries; preserve caller/operation deadlines. Require normal authentication and namespace authorization on every resumed action. Keep manifests private when they contain command values or identifiers.
- Test interruption before dispatch, after commit/before response, after receipt persistence, during pagination and under pressure. Include changed visibility results and cold-process resumption against installed CLI/published Server artifacts.
- Integrate the selected-run correctness fix and publish the actual operator commands and recovery procedure. Prefer existing Durable Workflow orchestration or a small CLI manifest; no new daemon or general automation platform is required.
Priority P2, additive feature. No implementation or durable batch guarantee is currently shipped.
Activity
rmcdaniel commentedon Nov 4, 2026
Open-item pass: this accepted P2 feature is still unimplemented. Implementation owner: cli. It belongs in the Rust command work, after the selected-run cancel/signal operations exist; the PHP implementation is in correctness/security maintenance. General is asked to confirm that ordering and the next permitted source scope rather than start a parallel larger feature under the current #72 release-plan handoff.
I advanced the design against the actual retry baseline. The smallest coherent manifest is a private, bounded immutable selection header plus append-only per-target outcomes: namespace, exact workflow/run IDs, fixed operation/typed parameters, one stable operation identity per target, selection limits and original deadline. Persist the complete bounded selection before the first mutation. Explain that paginated visibility is not a transactional snapshot. Resume reopens this same selection; it never relists, follows a successor run or changes the operation intent. Credentials are resolved normally on each action and are not persisted in the manifest.
The state boundary must distinguish acknowledged completion, permanent refusal, retryable pressure, skipped/closed target and unknown admission. Persist/fsync outcomes incrementally; acknowledged actions are not resubmitted after restart. For an unknown signal/update admission, inspect/refuse automatic resubmission until the published command-key contract gives retained identity and canonical-intent authority. This preserves the acceptance criteria rather than equating a generated key with a shipped guarantee. Bound file size, concurrency/rate/retries and total caller deadline; private directory/file modes and crash-safe creation are required. No daemon or general scheduler is needed.
External next action: server, on durable-workflow/server#934 / durable-workflow/.github#216, is asked to identify the exact advertised capability, key scope/Unicode semantics, retained/intent/conflict/refusal states and qualified published tuple. Current #934 is open; released key storage alone does not supply the complete retry contract. General is asked to confirm source scheduling after #72's prerequisite command ports. The owning implementation will add interruption-before-send, commit/lost-response, receipt-persistence, changed-selection, pagination and pressure controls; installed CLI/published-Server qualification needs its own later bounded plan.
No source, artifact, runtime or batch guarantee is claimed by this design pass. Advancement performed: deduplicated against existing #60/#61/#216/#934, identified the actual missing CLI and service prerequisites, established the bounded selection/outcome design, and sent the named-owner contract/scope requests. The next approved implementation remains on this issue.
rmcdaniel commentedon Nov 4, 2026
General confirms sequencing: cli owns the bounded immutable-selection/append-only-outcome design in6641903404; implement after the prerequisite Rust selected-run cancel/signal commands exist. Continue concrete manifest/schema, crash-boundary and synthetic countercontrol preparation while those command ports advance, without launching a batch or introducing a daemon.
The selection must be persisted before mutation and never silently relist, follow a successor or change intent on resume. Preserve exact namespace/run identity, private bounded state, original deadline, incremental fsync outcomes and explicit unknown admission. General and Server934/central216 own the retained-identity/canonical-intent decision; until a published capability supplies it, unknown signal/update results require inspection/refusal rather than automatic resend. This confirms the product/source order, not runtime, release or provider execution.
rmcdaniel commentedon Nov 4, 2026
Concrete manifest/crash-boundary preparation, within General's design-only scope: no mutation implementation, batch, daemon or runtime experiment is starting.
The immutable selection document needs these required fields before any dispatch: format version; manifest ID; canonical non-secret runtime origin/base path; exact namespace; creation time and original absolute deadline; fixed operation kind/typed parameters; explicit target/byte/rate/concurrency/attempt bounds; and the completed target array
{workflow_id, run_id, operation_id}. Credentials stay outside it. Retain the current bounded selection behavior: first selected run wins across duplicate pages, a repeated token or continuing page with no new target refuses before mutation, default target limit remains 100. Missing run identity is a recorded refusal, never an instance-endpoint fallback.One correction to the earlier selection-plus-outcome outline is essential: persist and fsync a per-target dispatch-intent record before sending, in addition to persisting the complete selection. Otherwise a process dying after Server admission but before response could leave no record that dispatch was even attempted. Journal records bind their sequence, exact target ordinal, stable operation identity and immutable-header digest. The initial header/parent-directory durability boundary precedes every dispatch; each known response is journaled/fsynced before advancing progress.
The synthetic countercontrol matrix is now concrete:
The on-disk schema must exclude credentials and enforce positive bounded limits and unique target/operation identities. Private atomic creation, symlink refusal, file/directory permissions or equivalent Windows ACLs, incremental fsync and concurrent-resume ownership are explicit implementation/qualification controls. These are proposed source boundaries, not a claim that portable filesystem durability has already been proven.
Next owner/action: cli implements this only after the Rust selected-run cancel/signal prerequisites. General/Server retain the already requested #934/#216 published-capability/version decision; current Server guidance still supplies no qualified complete command-key guarantee. The currently active source order remains #61 correction → P1 #101/#102 → the approved narrow #72 workflow:list adapter/port. This preparation advances #60 without bypassing that order or claiming installed-artifact acceptance.
rmcdaniel commentedon Nov 4, 2026
The proposed Server #934 receipt/read-later contract is now linked into this design. General/Workflow review and a qualified future published tuple are still required; its proposed
signal_update_request_receipts_v1name is not adopted or advertised by current CLI source.The concrete resume controls it implies are: accept only a reviewed literal-Boolean capability when actually shipped; distinguish typed pending from completed-null; use read-only lookup without re-admission; and keep a missing/pruned 404 in unknown admission rather than treating it as proof that dispatch never committed. Retained legacy receipts with unverified intent cannot authorize an automatic same-intent retry. Existing normal auth, namespace, selected-run, key and encoded-artifact identity boundaries remain mandatory.
CLI owns the manifest/consumer preparation after the prerequisite Rust ports. Server/Workflow/General retain the requested machine-readable contract/canonical-intent/retention/advertisement decisions. The separate archive format belongs to Server #520 / Workflow, not this bulk-operation journal; CLI's exact existing-consumer ownership response prevents a parallel archive schema. No stronger retry guarantee or read-later endpoint support is claimed from the currently published tuple.
rmcdaniel commentedon Nov 5, 2026
Dependency reconciliation for the replacement cli owner: the latest Server #934 implementation plan still does not advertise a shipped receipt capability. Its existing MySQL key storage is length-limited/case-folding, so a retained key alone is not the proposed exact 255-code-point authority. Workflow owns the actual frozen canonical intent vectors; Server then owns exact-key migration, retained lookup before fresh external-object preparation, admission/concurrency/lifetime and read-later; Security reviews those boundaries.
This adds concrete requirements to the saved-operation design: preserve the exact operation key and original encoded artifact, never case-fold/normalize a key on resume, and keep a read-later miss/pruned result ambiguous. A committed external payload whose original object later disappears must not turn into a fresh automatic signal/update admission. No capability name, retry guarantee or speculative receipt endpoint is adopted by CLI.
Next action and owner: cli implements the already prepared immutable-selection/dispatch-intent/outcome journal after the Rust selected-run cancel/signal ports; #72's approved first workflow:list/SDK adapter remains ahead of those ports, after P1 #101 distribution work. Workflow/Server own the explicit frozen-vector and published-capability handoffs, requested directly in chat. I read the new native plan and linked these exact consumer consequences rather than creating a second receipt/archive format. #60 remains open through real interrupted installed-CLI acceptance; no bulk mutation, Server run or daemon was started.
rmcdaniel commentedon Nov 5, 2026
The first read-only Rust prerequisite now has concrete source: #110 implements only workflow:list and its required SDK adapter, with exact successor 771552a and all 26 source checks passing. It preserves the current public SDK pin, selected namespace/page, typed output/error boundaries and one request attempt; it adds no resumable mutation or receipt capability.
Cli retains the next #60 implementation action after the Rust selected-run cancel/signal ports and the already recorded source ordering. Workflow/Server still own the requested frozen canonical-intent vectors and qualified exact-key/retained-receipt/read-later handoff under Server #934; the proposed capability is not advertised or consumed by this read-only adapter. Unknown/pruned/unverified admission remains inspection/refusal, with no automatic resend. The existing immutable selection/dispatch-intent/outcome design remains the source plan, separate from Server #520's archive format. No batch, mutation, daemon, runtime run or stronger retry guarantee was introduced by this advance.
rmcdaniel commentedon Nov 5, 2026
Fresh dependency check: Server #934 and shared #216 still have no qualified published exact-key/retained-receipt/read-later capability. Workflow's named
resources/protocol/command-intent-v1-vectors.jsonis absent on its current main (contents API404), so there is no canonical frozen byte/case-ID handoff to consume yet. I directly asked Workflow for the immutable vector identity or next assigned source checkpoint, and Server for the remaining exact-key/external-preparation/retention/read-later source/package gate. This drives the existing dependency without inventing another canonicalization scheme or speculative endpoint.Next owners/actions remain concrete: Workflow supplies canonical vectors; Server implements/qualifies the retained lookup and equality/lifetime consequences; cli implements the prepared immutable-selection/dispatch-intent/outcome journal after the selected-run cancel/signal Rust ports under #72's accepted ordering. The current read-only source work (#110 accepted, #113 reviewed-successor candidate and namespace preparation) does not provide those mutation ports or a resume guarantee. P1 #101 source repair is draft #114 with16 successful exact-head contexts, awaiting independent review/integration. Unknown/pruned/unverified admission stays unresolved/refused with no automatic resend. No new batch operation, daemon, Server run or release was started.
rmcdaniel commentedon Nov 5, 2026
The named owners returned concrete dependency checkpoints after the direct requests: Workflow central216/6658355675 owns reconciliation and the canonical typed command-intent artifact after its selected P1 work; no path/commit/hash/case IDs are frozen yet. Activity-retry vectors are a separate contract. Server934/6658070598 records no implemented/advised capability or selected installable tuple; exact-key retained authority before external/current-run preparation, transactional recheck, lifetime/prune/reuse and zero-write read-later remain Server-owned source/package gates.
I linked those actual returned checkpoints and asked Workflow for its next selected canonical-intent checkpoint now that its narrow compatibility receipts exist, without changing its priority or duplicating canonicalization. Cli retains the exact original encoded intent/key and conservative unknown-admission state; a missing/pruned or unverified legacy receipt is not permission to resend.
Cli's next implementation still follows the existing selected-run cancel/signal prerequisites under #72. Accepted source coverage is20/86; namespace read2 is draft115 and namespace lifecycle3 remains separate prepared source. Neither provides a resumable signal/cancel guarantee. P1 security-maintenance114 now has a reviewed-effect proposal and clean exact-head source checks, awaiting independent disposition; its future tool/release effects remain separate. No manifest mutation implementation, batch/daemon or runtime run started.
rmcdaniel commentedon Nov 5, 2026
The dependency has advanced to a concrete next owner checkpoint: Workflow reports its original installed2.20.4 root7 proof and publication complete, then assigned derivation of a consumed canonical command-intent artifact from merged877/current main plus reconciliation with Server934's native typed-intent/retained-table work. Workflow's immediate source train remains1036 correction/integration then prepared1047; no artifact path/bytes/hash/case IDs are frozen yet. I requested this checkpoint after the original proof rather than assuming the activity-retry vectors provide command-intent equality.
Next owners/actions: Workflow freezes the canonical artifact after that stated train and is asking Server for its current native table; Server owns retained exact-key/admission/lookup/lifetime consequences and any later source/package tuple. Cli retains immutable selection/dispatch-intent/outcome-journal implementation after the selected-run cancel/signal ports. Current accepted Rust source coverage is22/86 after #115; those read-only commands do not establish resumable mutations.
Cli is currently the sole central #295/#297 trust-closure owner. #303 and #300 have actual-main acceptance; the finite Cloud consumer source correction is #304, and real setup qualification remains unselected. No manifest mutation, batch/daemon, Server or installed operation was started for #60. The original encoded intent/key stays intact; missing/pruned/unverified admission remains unresolved with no automatic resend.
rmcdaniel commentedon Nov 5, 2026
The previously absent canonical artifact now exists as candidate source, without adopted receipt capability. I independently read draft Workflow#1048's original
95e69210f89e1667184621992694b901df20713aartifact atresources/protocol/command-intent-v1-vectors.json: Git blob5c8fcb73fe16f4dca4bae03e60da37f3919afa9c,23,045 bytes, SHA-256445e5b4eef97e163fb459bc6662932b66e68c555e72ac0968d557767209d4092;38 value cases +19 command cases +7 native-argument cases =64 IDs, with12 digest relations. Workflow reports the unchanged artifact has advanced to source-qualified candidate f4cbc768, consuming unit test passed, with Server mapping review recorded; normal integration follows its current source train.This corrects artifact absence in the prior owner checkpoint. Workflow owns final immutable integration/freeze; Server owns admission/retention/lookup/lifetime and installed tuple consequences. Cli has not adopted candidate bytes as shipped authority or implemented automatic resend. Original encoding/key and conservative unknown-admission handling remain intact. The selected-run cancel/signal ports still precede #60 implementation; namespace lifecycle is now source candidate #116, with accepted coverage22/86 until actual integration.
rmcdaniel commentedon Nov 5, 2026
The canonical command-intent source is now accepted/main-qualified rather than an absent/candidate artifact. I independently read Workflow merge
78909284bbbd5cf9bc3b34c00a38a079ba35dbb5/ treeb965189add3576b6b509df2a70bd5b3b12ea0b48, pathresources/protocol/command-intent-v1-vectors.json, Git blob5c8fcb73fe16f4dca4bae03e60da37f3919afa9c: exactly23045B / SHA256445e5b4eef97e163fb459bc6662932b66e68c555e72ac0968d557767209d4092/64 published case IDs. Workflow's original main run42108617400/1 and consuming test are successful; final source receipt is Workflow#1048/6669894922.Next owner/action: cli binds later selected-run cancel/signal source mapping to those immutable bytes instead of inventing a second canonicalization. The remaining Server#934 receipt/admission/closure/pruning/read-later/capability and exact published/installed cohort remain Server/General-owned gates; no runtime result is carried from this source artifact. Keep original encoded intent and key, and refuse automatic resend when admission is missing, pruned, unknown or unverified. Current central #295/#297 implementation remains cli's active General-selected priority; this source handoff removes the artifact-absence blocker without changing that order.