refactor preview by JoshC8C7 · Pull Request #1454 · doublewordai/control-layer

JoshC8C7 · 2026-08-07T10:57:33Z

No description provided.

… from onwards yet.

Move the Responses edge translation from the outermost layer to inner of the outlet, and make the inference middleware outermost, so GET /v1/responses/{id} and background mode work. The outlet now persists the translated Responses object (what GET reads); the control plane (id minting, previous_response_id hydration, background/flex routing) owns the Responses lifecycle in the inference middleware; and the translator becomes a pure converter that stamps the platform tracking id so client and stored ids match. Also adds /messages to should_intercept and a POST-then-GET test for the /responses path.

…wctl (COR-536) With Responses translation placed correctly at the dwctl edge, onwards' Responses adapter is dead and the server-side multi-step tool loop is retired (COR-517). Removes the warm-path loop dispatch, the engine loop-driver, the MultiStepStore impl + loop storage helpers, the multi-step processor wiring, and HttpToolExecutor - keeping flex/background/GET, the storage substrate, and the tool-injection primitives. Relocates ResponseStore/NoOpResponseStore/StoreError from onwards into dwctl (its only remaining user). Pairs with the onwards deletion PR.

dwctl's inference_middleware now strips client-supplied id/completion_id/response_id keys from the request body before it re-serialises, so the scrub (onwards PR #240) happens where dwctl owns the single parse-and-shape. Exact-key removal preserves previous_response_id and every other extension. Removes the dead ported scrub from responses/util.rs + types.rs. Lets onwards forward the bytes verbatim.

# Conflicts: # Cargo.lock # dwctl/Cargo.toml # dwctl/src/inference/middleware.rs

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

The blanket POST gate added for the /responses review comment also blocked AnthropicModels, which deliberately claims GET /models to normalise x-api-key into Authorization. That 401'd Anthropic model discovery. Move the gate behind translator.translates_request_body(), so it only guards translators that actually deserialise a body. Defaults to true; AnthropicModels overrides it to false.

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

The multi-step feature on onwards pulls in the optional fusillade dep, which the merged lockfile was missing.

…billing onto one parse Fixes Anthropic under-charging: Anthropic responses had no deserialize-able type so they fell to AiResponse::Other and billed zero tokens. Logging and billing now derive from a single parse_ai_response -> AiResponse (via TokenMetrics::from) instead of two parses through divergent types. Responses parse with dwctl's own types.rs; Anthropic with model.rs (now Deserialize + Clone); OpenAI endpoints keep async-openai (typed reasoning tokens, tolerant of raw passthrough bodies). Removes the UsageExtractor / ProtocolUsage / UsageRegistry layer and the per-translator extract_usage impls. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

Formatting-only fixup for test helpers added in the previous commit before the final fmt pass ran. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

onwards' strict chat/completions/embeddings handlers now validate the request shape and forward the ORIGINAL bytes instead of deserialising and re-serialising (which reordered keys and silently dropped unknown nested fields). The caller id-scrub and the streaming usage-flag injection (stream_options.include_usage, x-fusillade-stream force-stream) move into a new dwctl outbound_request middleware, wired innermost on the onwards router (inner to cache). This retires dwctl's stream_usage_transform + the onwards BodyTransformFn wiring, so onwards no longer manipulates request bodies for edge traffic. The /responses background strip is dropped: translation has already flattened /responses to chat by that layer. Malformed-request status mapping (422 for schema-invalid, 400 for bad JSON) is preserved via a shared parse helper. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

# Conflicts: # dwctl/src/inference/engine/assembly.rs # dwctl/src/inference/engine/loop_http_client.rs # dwctl/src/inference/engine/processor.rs # dwctl/src/inference/engine/transition.rs # dwctl/src/inference/middleware.rs # dwctl/src/inference/store.rs # dwctl/src/inference/streaming.rs # dwctl/src/lib.rs # dwctl/src/request_logging/serializers.rs # dwctl/src/test/multi_step_executor.rs # dwctl/src/test/responses.rs

…the body When the streaming usage flags are injected the body changes size, so the inbound Content-Length no longer matches. onwards forwards headers verbatim to the upstream, so a stale length could truncate or hang the upstream read. Remove the header (as the Anthropic translator already does) so it is recomputed. Addresses a Copilot review comment on PR #1317. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

After the multi-step machinery was removed on this branch, dwctl no longer references onwards-fusillade (main still declares it because main uses the multi-step loop). Removing the unused path dependency. Addresses a Copilot review comment on PR #1317. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

This reverts commit 5d11e27.

# Conflicts: # dwctl/src/inference/store.rs # dwctl/src/request_logging/analytics_handler.rs

# Conflicts: # dwctl/src/test/strict_mode.rs

Copilot

Pull request overview

This PR refactors the “strict mode” request/response flow so that onwards becomes a schema validator + forwarder (avoiding request-body re-serialization), while dwctl takes ownership of request-body mutation and OpenAI Responses translation, and updates logging/billing parsing to use dwctl-owned response types.

Changes:

Move strict-mode chat/completions/embeddings in onwards to “validate then forward original bytes” and remove request-id scrubbing from strict schemas.
Add a dwctl OpenAI Responses edge translator (request/response + streaming + previous_response_id hydration) and adjust translation middleware to pass original request bytes + tracking id into translators.
Update dwctl request-logging/billing parsing to use dwctl-owned Responses + Anthropic types, remove legacy stream-usage transform, and simplify/replace several tests and retired multi-step loop artifacts.

Reviewed changes

Copilot reviewed 43 out of 44 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
onwards/src/strict/schemas/utils.rs	Removes request-id scrubbing helper from strict schema utilities.
onwards/src/strict/schemas/responses.rs	Drops request-id scrubbing method/import from ResponsesRequest.
onwards/src/strict/schemas/chat_completions.rs	Drops request-id scrubbing method/import from ChatCompletionRequest.
onwards/src/strict/handlers.rs	Strict handlers now validate shape and forward original bytes; updates tests accordingly.
dwctl/src/test/strict_mode.rs	Adjusts strict-mode integration test polling for 403/404 startup races.
dwctl/src/test/responses.rs	Replaces multi-step store-based test with POST-then-GET by client id test.
dwctl/src/test/multi_step_executor.rs	Removes end-to-end integration test for retired multi-step loop executor.
dwctl/src/test/mod.rs	Removes multi_step_executor module export.
dwctl/src/request_logging/utils.rs	Parses Responses/ResponsesStream via dwctl-owned types; adds Anthropic parsing helpers.
dwctl/src/request_logging/stream_usage.rs	Removes old request-body stream-usage transform module.
dwctl/src/request_logging/models.rs	Extends AiResponse to include dwctl-owned Responses + Anthropic variants.
dwctl/src/request_logging/mod.rs	Removes stream_usage module export.
dwctl/src/request_logging/analytics_handler.rs	Simplifies parse/zero-token alarming; gates alarms by usage-bearing paths.
dwctl/src/metrics/errors.rs	Removes unused ZDR_DISPATCH component constant.
dwctl/src/inference/translation/responses/util.rs	Adds helper functions supporting Responses translation (ported/duplicated).
dwctl/src/inference/translation/responses/response.rs	Implements Chat Completions → Responses response conversion.
dwctl/src/inference/translation/responses/request.rs	Implements Responses → Chat Completions request conversion.
dwctl/src/inference/translation/responses/mod.rs	Adds OpenResponses translator with request/response + streaming reframing.
dwctl/src/inference/translation/responses/hydrate.rs	Adds previous_response_id hydration step using ResponseStore.
dwctl/src/inference/translation/mod.rs	Expands translation trait to pass original request + tracking id; adds Responses module.
dwctl/src/inference/translation/middleware.rs	Adds per-translator POST gating and forwards request bytes + tracking id into translators.
dwctl/src/inference/translation/anthropic/models.rs	Marks models translator as header-only (no body) and updates trait impl.
dwctl/src/inference/translation/anthropic/model.rs	Makes MessagesResponse (and related enums) Deserialize for logging/billing.
dwctl/src/inference/translation/anthropic/mod.rs	Updates Anthropic translator impl for new trait signature.
dwctl/src/inference/tools/mod.rs	Updates module docs/exports to reflect removal of server-side executor use.
dwctl/src/inference/tools/executor.rs	Retires HttpToolExecutor implementation; keeps resolved tool rendering helpers.
dwctl/src/inference/streaming.rs	Removes warm-path loop streaming; keeps flex replay SSE helper.
dwctl/src/inference/store.rs	Removes multi-step loop side-channel + MultiStepStore implementation; keeps ResponseStore wiring.
dwctl/src/inference/response_store.rs	Introduces dwctl-local ResponseStore trait (relocated from onwards).
dwctl/src/inference/outbound_request.rs	Adds last-mile middleware for stream usage-flag injection before onwards.
dwctl/src/inference/mod.rs	Wires new outbound_request + response_store modules into inference.
dwctl/src/inference/handler.rs	Switches StoreError import to dwctl-local response_store.
dwctl/src/inference/engine/processor.rs	Removes daemon processor for retired multi-step loop.
dwctl/src/inference/engine/mod.rs	Narrows engine module to persistence (writer/outlet_handler).
dwctl/src/inference/engine/loop_http_client.rs	Removes loop-backed fusillade HttpClient implementation.
dwctl/src/inference/engine/assembly.rs	Removes loop-chain assembly implementation.
dwctl/src/api/models/requests.rs	Extends API response wrapper for Anthropic response variants.
.sqlx/query-c29f0d9ae97f622eb24178d88c93fff72be81635a89ee33e0a8a5385d5655fed.json	Removes SQLx prepared query metadata for deleted tool_call_analytics insert.

Files not reviewed (1)

.sqlx/query-c29f0d9ae97f622eb24178d88c93fff72be81635a89ee33e0a8a5385d5655fed.json: Generated file

Suppressed comments (2)

onwards/src/strict/handlers.rs:423

resolved_model is computed by re-parsing the request body (extract_model_from_request(&headers, &body_bytes)), which is redundant because the body has already been deserialized into EmbeddingsRequest. Checking model-override directly avoids an extra JSON parse on every embeddings call.
onwards/src/strict/handlers.rs:500
resolved_model is computed by calling extract_model_from_request(&headers, &body_bytes), which re-parses JSON that has already been deserialized into CompletionRequest. Since the only precedence logic is the model-override header, this can be implemented without a second parse.

    let resolved_model =
        extract_model_from_request(&headers, &body_bytes).unwrap_or(original_model.clone());


+    // Outer layers (onwards body limit, cache) already bound the body, so buffering
+    // with no extra limit here can't widen the exposure.
+    let bytes = match axum::body::to_bytes(body, usize::MAX).await {
+        Ok(b) => b,
+        Err(_) => return (StatusCode::BAD_REQUEST, "failed to read request body").into_response(),


+    // Last-mile request-body prep, applied innermost so it runs right before onwards
+    // (inner to cache, which must hash the original body). Does the id-scrub and the
+    // streaming usage-flag injection that onwards / the BodyTransformFn hook used to
+    // do, so onwards can forward the body untouched.


+//! `ensure_field` / `scrub_request_id_fields_from_extra` are copied from onwards'
+//! `strict::schemas::utils`; `merge_reasoning_text` / `chat_usage_to_response_usage`
+//! from onwards' `strict` module (both `pub(crate)` there, so not importable).
+//! They are duplicated here as part of moving Responses ownership into dwctl; the
+//! onwards copies retire with the rest of its Responses code (COR-536).
+
+// `ensure_field` / `scrub_request_id_fields_from_extra` back the copied response
+// normaliser, which dwctl doesn't call yet; allow dead code until it's wired.


@@ -314,25 +311,6 @@ mod tests {
    use std::time::{Duration, SystemTime};
    use tokio::sync::mpsc;


cloudflare-workers-and-pages · 2026-08-07T11:27:36Z

Deploying control-layer with Cloudflare Pages

Latest commit:	`0227675`
Status:	✅ Deploy successful!
Preview URL:	https://809da381.control-layer.pages.dev
Branch Preview URL:	https://preview-responses-edge-in-dw-p0on.control-layer.pages.dev

View logs

# Conflicts: # dwctl/src/inference/engine/loop_http_client.rs # dwctl/src/inference/middleware.rs # dwctl/src/inference/streaming.rs

JoshC8C7 and others added 29 commits July 16, 2026 15:13

v1 - copy response handling to dwctl and switch over, but dont delete…

147050e

… from onwards yet.

Merge branch 'main' into josh/responses-edge-in-dwctl

f141e51

# Conflicts: # Cargo.lock # dwctl/Cargo.toml # dwctl/src/inference/middleware.rs

lockfile

f18d283

comments

8e16ccc

Merge branch 'main' into josh/responses-edge-in-dwctl

efa5bb9

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

onwards

4d851ca

comments

04239d2

Merge branch 'main' into josh/responses-edge-in-dwctl

0f7f3c5

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

chore: sync Cargo.lock after merging main

b7d6a47

The multi-step feature on onwards pulls in the optional fusillade dep, which the merged lockfile was missing.

Merge branch 'main' into josh/responses-edge-in-dwctl

b7782f8

comments

3bd892e

Merge branch 'main' into josh/responses-edge-in-dwctl

b564c65

Merge branch 'main' into josh/responses-edge-in-dwctl

fd8db2b

translate in non-strict mode also

e77c29e

style: rustfmt anthropic billing test helpers

76dbde4

Formatting-only fixup for test helpers added in the previous commit before the final fmt pass ran. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

sqlx

0b372d2

Revert "chore(dwctl): drop unused onwards-fusillade dependency"

fc8555b

This reverts commit 5d11e27.

Merge branch 'main' into josh/responses-edge-in-dwctl

368a12b

# Conflicts: # dwctl/src/inference/store.rs # dwctl/src/request_logging/analytics_handler.rs

Merge branch 'main' into josh/responses-edge-in-dwctl

624481a

Merge branch 'main' into josh/responses-edge-in-dwctl

a7a0e15

# Conflicts: # dwctl/src/test/strict_mode.rs

Copilot AI review requested due to automatic review settings August 7, 2026 10:57

Copilot started reviewing on behalf of JoshC8C7 August 7, 2026 10:58 View session

Copilot AI reviewed Aug 7, 2026

View reviewed changes

Merge branch 'main' into josh/responses-edge-in-dwctl

c5ad27d

# Conflicts: # dwctl/src/inference/engine/loop_http_client.rs # dwctl/src/inference/middleware.rs # dwctl/src/inference/streaming.rs

JoshC8C7 force-pushed the preview/responses-edge-in-dwctl2 branch from 0227675 to c5ad27d Compare August 10, 2026 10:34

Navigation Menu

Uh oh!

refactor preview - #1454

refactor preview#1454
JoshC8C7 wants to merge 30 commits into
mainfrom
preview/responses-edge-in-dwctl2

JoshC8C7 commented Aug 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		let resolved_model =
		extract_model_from_request(&headers, &body_bytes).unwrap_or(original_model.clone());

		@@ -314,25 +311,6 @@ mod tests {
		use std::time::{Duration, SystemTime};
		use tokio::sync::mpsc;

Uh oh!

Conversation

JoshC8C7 commented Aug 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 7, 2026

Deploying control-layer with Cloudflare Pages

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants