Skip to content

fix(server): unify generation failure responses - #689

Open
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/serving-result-boundary
Open

fix(server): unify generation failure responses#689
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/serving-result-boundary

Conversation

@Graffioh

@Graffioh Graffioh commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a server-owned ResponseError boundary that maps typed backend failures once
  • stop classic generation failures before success telemetry, cache memory, SSE completion, or HTTP 200 response construction
  • emit native OpenAI, Anthropic, and Responses API failure shapes for streaming and non-streaming clients
  • replace the scheduler's separate failure flag and error string with one optional error value
  • make success and failure terminals mutually exclusive in SseEmitter

Design

This keeps the current execution boundaries intact:

  • ModelBackend::generate() remains the classic whole-request interface
  • SeqEngine::step() remains the concurrent iteration interface
  • GGML and backend code do not receive HTTP, JSON, or SSE types
  • socket ownership stays in the classic worker and scheduler

The PR deliberately keeps ParsedRequest, GenerateRequest, GenerateResult, ServerJob, SchedSlot, and DaemonIO. It introduces a name only for the missing server-side failure consequence instead of renaming the pipeline or adding a common generation hierarchy.

This is complementary to #688 and does not touch its backend-configuration files.

Client behavior

  • OpenAI-compatible streams receive an error object followed by [DONE]
  • Anthropic streams receive an error event and no success message_stop
  • Responses streams receive response.failed and no response.completed
  • non-streaming failures use a mapped 400, 500, or 503 status with the matching API error envelope

Verification

  • C++17 syntax checks pass for response_error.cpp, sse_emitter.cpp, http_server.cpp, and scheduler.cpp
  • a model-free executable built from the production response-error and SSE sources passes all three protocol failure cases and terminal exclusivity
  • git diff --check passes
  • full GPU-linked test_server_unit is left to CI because the local host has neither CUDA nor HIP

Review in cubic

@Graffioh
Graffioh marked this pull request as ready for review September 2, 2026 12:27

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread server/src/server/response_error.cpp
Comment thread server/src/server/http_server.cpp
Comment thread server/test/test_server_unit.cpp Outdated
@Graffioh
Graffioh marked this pull request as draft September 2, 2026 12:41
@Graffioh
Graffioh marked this pull request as ready for review September 2, 2026 19:14

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files

Requires human review: Auto-approval blocked by 3 unresolved issues from a previous review of this commit.

Re-trigger cubic

@Graffioh
Graffioh force-pushed the codex/serving-result-boundary branch from 054eaa0 to 0c0d312 Compare September 3, 2026 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant