Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/diary/how-to-build-modules.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[unrelated bug] `docs/guides/getting-started.md` declares `const { quotes } = service.load();` twice in the gateway server example. This makes the copied example fail to compile and should be fixed in a separate documentation task.
34 changes: 34 additions & 0 deletions .agents/diary/queues-prototype.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Queue prototype diary

- **Unrelated bug.** `prisma-composer dev` printed React "Invalid hook call"
warnings twice before all local services became ready. The queue application
still ran correctly, so investigating the CLI rendering issue is outside this
prototype.

- **Scope tangent.** Composer should validate that `PRISMA_WORKSPACE_ID` matches
the service token workspace before creating a project. A mismatch can make
project discovery miss an existing project while creation still succeeds in
the token workspace, producing an unwanted duplicate project.

- **Scope tangent.** The deployed demo relies on the `--name queues-demo`
override while its root Module is named `queues-example`. A later deploy that
omits the override creates a second project; the example should make its
stable production name harder to omit.

- **Unrelated bug.** A deploy that changed an artifact together with its input
document or dependency URL started the new artifact before the new binding was
active. The queue service entered a restart loop with its previous input, and
the dispatcher called previous queue and consumer routes. Reconciliation after
the bindings settled fixed both cases; deployment ordering should prevent this
state.

- **Scope tangent.** Promoting a new Compute deployment leaves every previous
deployment running. This is unsafe for continuously polling drivers: six queue
dispatcher revisions competed for work, and older revisions used stale
bindings. Composer needs deployment retirement or a revision-fencing mechanism
before always-running drivers are production-safe.

- **Unrelated bug.** The queue specification's relative links still use its old
`projects/queue-module/` depth after the file moved under `.drive/projects/`.
They currently resolve below `.drive/` instead of the repository `docs/`
directory and should be corrected separately.
702 changes: 702 additions & 0 deletions .drive/projects/queue-module/spec.md

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions .drive/projects/queue-module/trace.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{"event_id":"3624df39-5574-4c3c-b3e3-1a1f7e388b78","schema_version":"1","ts":"2026-07-31T15:50:52.276Z","project_run_id":"queue-module","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":".drive/projects/queue-module/spec.md","spec_kind":"project","byte_length":41966,"bytes_delta":378,"edge_cases_count":null,"open_questions_count":0,"dod_items_count":0,"reason":"scope-shift","sections_changed":["Summary","Prototype status","Context","At a glance","Approach","Functional Requirements","Non-Functional Requirements","Non-goals","Acceptance Criteria"]}
{"event_id":"87eae25d-6ad6-48ea-aac2-e315a9390bcb","schema_version":"1","ts":"2026-07-31T16:39:12.216Z","project_run_id":"queue-module","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":".drive/projects/queue-module/spec.md","spec_kind":"project","byte_length":43142,"bytes_delta":1176,"edge_cases_count":null,"open_questions_count":0,"dod_items_count":0,"reason":"operator-correction","sections_changed":["Approach","Functional Requirements","Acceptance Criteria"]}
{"event_id":"6e68fa34-5834-4857-8496-7c1ad1f805fb","schema_version":"1","ts":"2026-07-31T16:44:37.730Z","project_run_id":"queue-module","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":".drive/projects/queue-module/spec.md","spec_kind":"project","byte_length":44014,"bytes_delta":872,"edge_cases_count":null,"open_questions_count":0,"dod_items_count":0,"reason":"scope-shift","sections_changed":["Summary","Context","Approach","Non-goals"]}
{"event_id":"f020ea47-a748-419c-825d-7cacb0bcefed","schema_version":"1","ts":"2026-08-01T09:30:19.285Z","project_run_id":"queue-module","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":".drive/projects/queue-module/spec.md","spec_kind":"project","byte_length":42899,"bytes_delta":-1115,"edge_cases_count":null,"open_questions_count":0,"dod_items_count":0,"reason":"operator-correction","sections_changed":["Prototype status","Approach","Functional Requirements","Non-Functional Requirements","Non-goals","Acceptance Criteria"]}
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ npx skills add prisma/composer --skill prisma-composer
That's the whole setup. Your agent now knows the entire API and arrives
prepped with the **building blocks** it can snap together — ready-made
Modules for scheduled jobs, blob storage, and event streams, alongside the
ones you write. Ask it for what you want ("a Next.js storefront calling an
ones you write. Persistent work queues are available as an early prototype.
Ask it for what you want ("a Next.js storefront calling an
orders API with its own Postgres, deployed to a staging stage") and let it
compose.

Expand Down Expand Up @@ -131,7 +132,7 @@ have.
| Guide | Covers |
| --- | --- |
| [Getting started](docs/guides/getting-started.md) | Your first app end to end; porting an existing Node or Next.js app |
| [Building an app](docs/guides/building-an-app.md) | Contracts, databases (plain + Prisma Next-typed with migrations), reusable Modules, cron/storage/streams, config, secrets |
| [Building an app](docs/guides/building-an-app.md) | Contracts, databases (plain + Prisma Next-typed with migrations), reusable Modules, cron/storage/streams/queues, config, secrets |
| [Testing](docs/guides/testing.md) | Unit tests with `mockService`, integration tests with `bootstrapService` |
| [Deploying and operating](docs/guides/deploying.md) | Stages, destroy, CI, how apps behave in production |

Expand All @@ -147,14 +148,16 @@ Complete, deployable apps under [`examples/`](examples/):
| [cron](examples/cron/) | Scheduled jobs: `defineSchedule` + `serveSchedule` + the cron module |
| [storage](examples/storage/) | The S3-backed blob store module |
| [streams](examples/streams/) | Durable append-only event streams over storage |
| [queues](examples/queues/) | Persistent queue delivery from and back to one Compute service |

## Building blocks and extensions

A **Module** is the unit of reuse: it owns its internals and exposes a typed
port, so composing one is a couple of lines and never an integration you have
to invent. Three ship inside `@prisma/composer-prisma-cloud` today — `cron`
(scheduled jobs), `storage` (S3-backed blobs), and `streams` (durable event
streams) — alongside the Modules you write yourself.
to invent. Four ship inside `@prisma/composer-prisma-cloud` today — `cron`
(scheduled jobs), `storage` (S3-backed blobs), `streams` (durable event
streams), and the early `queues` prototype (Postgres-backed work delivery) —
alongside the Modules you write yourself.

An **extension** is a package that brings its own Modules, resources, or
deploy target. The convention is an npm package named `prisma-composer-*` —
Expand Down
48 changes: 48 additions & 0 deletions architecture.config.json
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,54 @@
"layer": "modules",
"plane": "execution"
},
{
"glob": "packages/1-prisma-cloud/2-shared-modules/queues/src/*.ts",
"domain": "prisma-cloud",
"layer": "modules",
"plane": "shared"
},
{
"glob": "packages/1-prisma-cloud/2-shared-modules/queues/src/execution/**",
"domain": "prisma-cloud",
"layer": "modules",
"plane": "execution"
},
{
"glob": "packages/1-prisma-cloud/2-shared-modules/queues/src/exports/index.ts",
"domain": "prisma-cloud",
"layer": "modules",
"plane": "shared"
},
{
"glob": "packages/1-prisma-cloud/2-shared-modules/queues/src/exports/queue-service.ts",
"domain": "prisma-cloud",
"layer": "modules",
"plane": "shared"
},
{
"glob": "packages/1-prisma-cloud/2-shared-modules/queues/src/exports/dispatcher-service.ts",
"domain": "prisma-cloud",
"layer": "modules",
"plane": "shared"
},
{
"glob": "packages/1-prisma-cloud/2-shared-modules/queues/src/exports/queue-entrypoint.ts",
"domain": "prisma-cloud",
"layer": "modules",
"plane": "execution"
},
{
"glob": "packages/1-prisma-cloud/2-shared-modules/queues/src/exports/dispatcher-entrypoint.ts",
"domain": "prisma-cloud",
"layer": "modules",
"plane": "execution"
},
{
"glob": "packages/9-public/composer-prisma-cloud/src/exports/queues.ts",
"domain": "public",
"layer": "public",
"plane": "shared"
},
{
"glob": "packages/9-public/composer/src/exports/deploy.ts",
"domain": "public",
Expand Down
54 changes: 53 additions & 1 deletion docs/guides/building-an-app.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
This guide covers everything you reach for once
[Getting started](getting-started.md) has shown you the shape: giving a
service a database (plain or Prisma Next-typed), packaging pieces as reusable
Modules, the cron/storage/streams modules that ship with the framework, and
Modules, the cron/storage/streams/queues modules that ship with the framework, and
the service input — configuration and secrets as one schema.

## How the pieces fit
Expand Down Expand Up @@ -315,6 +315,7 @@ is a couple of lines:
| `cron` from `@prisma/composer-prisma-cloud/cron` | A scheduler that fires your jobs at your service on an interval | nothing |
| `storage` from `@prisma/composer-prisma-cloud/storage` | An S3-backed blob store, credentials included | `store` |
| `streams` from `@prisma/composer-prisma-cloud/streams` | Durable append-only event streams, backed by a `store` | `streams` |
| `queues` from `@prisma/composer-prisma-cloud/queues` | Persistent work delivery backed by Postgres | `producer`, `dispatch` |

Cron is the one most apps want first. You supply two things — a schedule and
a runner service that exposes the `trigger` contract — and the module does
Expand Down Expand Up @@ -357,6 +358,57 @@ provision(cron({ schedule, runner: promotionsService }), {
[`examples/streams`](../../examples/streams/) show the other two, including
the streams module's secret binding.

The queues prototype uses one static catalogue for producer and consumer
typing. The queue Module owns its Postgres database and queue service. A
separate always-running dispatcher claims messages and pushes them to the
consumer service:

```ts
// queues.ts
import { defineQueues, fixedBackoff } from '@prisma/composer-prisma-cloud/queues';
import { type } from 'arktype';

export const appQueues = defineQueues({
thumbnails: {
message: type({ imageId: 'string' }),
retry: {
maxAttempts: 5,
delay: fixedBackoff({ delay: '5s' }),
},
},
});
```

```ts
// worker/service.ts
import { queueConsumer, queueProducer } from '@prisma/composer-prisma-cloud/queues';

export default compute({
name: 'worker',
deps: { queues: queueProducer(appQueues) },
expose: { consumer: queueConsumer() },
build: node({ module: import.meta.url, entry: '../../dist/worker/server.mjs' }),
});
```

```ts
// module.ts
const queue = provision(queues({ definitions: appQueues }));
const worker = provision(workerService, { deps: { queues: queue.producer } });

provision(queueDispatcher(), {
deps: { queue: queue.dispatch, consumer: worker.consumer },
input: { pollIntervalMs: 250, leaseSeconds: 30 },
});
```

Route `/rpc/*` in the worker to `serveQueues(workerService, appQueues,
handlers)`. A handler can enqueue follow-up work through the producer dependency,
including back to the same queue. This first prototype delivers one message per
request and supports per-queue fixed retry delays. Batches, exponential retry,
multiple competing consumers, replay, pause, and operational APIs remain future
work. See [`examples/queues`](../../examples/queues/) for the complete deployed app.

### Where new blocks come from

An **extension** is a package that brings its own Modules, resources, or
Expand Down
29 changes: 29 additions & 0 deletions examples/queues/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Queues demo

This deployed example proves one application Compute service can both produce
and consume persistent queue messages. Its browser sends messages through the
typed producer dependency. The separate dispatcher claims them from the queue
module's Postgres database and delivers them back to the same application
service's consumer port.

Each queue definition owns its retry policy. This demo uses a fixed five-second
delay and stops after five delivery attempts:

```ts
messages: {
message: type({ text: 'string' }),
retry: {
maxAttempts: 5,
delay: fixedBackoff({ delay: '5s' }),
},
}
```

Enable **Fail the first delivery** before enqueuing to exercise the real retry
path. The consumer throws on attempt one. The dispatcher stores the next
availability time in Postgres and delivers the same message again on attempt
two.

The consumed-message panel is a bounded in-memory demo feed. Queue durability
comes from Postgres; refreshing or restarting the application can clear the
display without losing queued messages.
13 changes: 13 additions & 0 deletions examples/queues/module.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import { module } from '@prisma/composer';
import { queueDispatcher, queues } from '@prisma/composer-prisma-cloud/queues';
import appService from './src/app/service.ts';
import { demoQueues } from './src/queues.ts';

export default module('queues-example', ({ provision }) => {
const queue = provision(queues({ definitions: demoQueues }));
const app = provision(appService, { deps: { queues: queue.producer } });
provision(queueDispatcher(), {
deps: { queue: queue.dispatch, consumer: app.consumer },
input: { pollIntervalMs: 250, leaseSeconds: 30 },
});
});
23 changes: 23 additions & 0 deletions examples/queues/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"name": "@prisma/example-queues",
"version": "0.3.0",
"private": true,
"type": "module",
"scripts": {
"build": "tsdown",
"typecheck": "tsc --noEmit",
"deploy": "pnpm turbo run build --filter @prisma/example-queues... && ( set -a; . \"${PRISMA_DEPLOY_ENV:-../../.env}\"; set +a; bun node_modules/.bin/prisma-composer deploy module.ts ${QUEUES_STACK_NAME:+--name \"$QUEUES_STACK_NAME\"} )",
"destroy": "( set -a; . \"${PRISMA_DEPLOY_ENV:-../../.env}\"; set +a; bun node_modules/.bin/prisma-composer destroy module.ts --production ${QUEUES_STACK_NAME:+--name \"$QUEUES_STACK_NAME\"} )"
},
"dependencies": {
"@prisma/composer": "workspace:0.3.0",
"@prisma/composer-prisma-cloud": "workspace:0.3.0",
"arktype": "^2.2.3"
},
"devDependencies": {
"@types/bun": "^1.3.13",
"tsdown": "^0.22.7",
"unrun": "^0.3.1",
"typescript": "^6.0.3"
}
}
8 changes: 8 additions & 0 deletions examples/queues/prisma-composer.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import { defineConfig } from '@prisma/composer/config';
import { nodeBuild } from '@prisma/composer/node/control';
import { prismaCloud, prismaState } from '@prisma/composer-prisma-cloud/control';

export default defineConfig({
extensions: [prismaCloud(), nodeBuild()],
state: prismaState(),
});
Loading