Skip to content

refactor(server): seal backend construction behind a plan - #691

Draft
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/backend-plan-boundary
Draft

refactor(server): seal backend construction behind a plan#691
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/backend-plan-boundary

Conversation

@Graffioh

@Graffioh Graffioh commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

This is the first implementation slice of #688. It makes backend construction consume one model-aware, immutable configuration instead of pairing a mutable BackendArgs with separately resolved facts.

  • make BackendArgs own operator input, including model and draft paths
  • replace the broad feature bag with a narrow BackendAdmissionContext containing only server-owned admission facts
  • resolve GGUF metadata, placement, compatibility, warnings, and SpecLA fallback into a move-only BackendPlan
  • make create_backend(BackendPlan) the only construction API; the old convenience and (args, plan) overloads are deleted
  • keep the plan alive in BackendRuntime because existing architecture configs retain path pointers after initialization
  • keep ServerConfig out of planning and construction; the HTTP layer projects only the admission facts the backend policy needs

The practical invariant is now enforced by the type boundary:

operator request + server admission facts
                  |
             prepare_backend
                  |
            BackendPlan (owned, normalized)
                  |
             create_backend
                  |
       BackendRuntime (plan + backend)

Why this slice

The current factory could validate one BackendArgs value and construct from another mutable value. Model paths were also borrowed C strings, while several backend configs retain those pointers beyond initialization. This change removes both failure modes without introducing a config-file language or model-profile inheritance.

SpecLA/DDTree selection and KVFlash fallback now happen during planning. Startup reporting and /props receive the same normalized values used for construction.

Tests

  • test_backend_plan: owned-path lifetime, construction API boundary, SpecLA normalization, KVFlash fallback, unsupported-model fallback, and missing-draft failure
  • test_feature_gate: existing compatibility and warning suite migrated to the narrower admission context
  • both lightweight suites pass locally with the CUDA placement contract

Next steps

This PR intentionally keeps the effective snapshot flat so the boundary can land as a reviewable unit. Follow-up work can:

  1. group BackendPlan fields by concern (placement, cache, speculation, and execution) once the existing callers are fully behind the plan
  2. move backend runtime reporting out of ServerConfig into a dedicated read-only info snapshot for /props
  3. add a small, explicit model-granular config loader for the supported model families; avoid profiles, inheritance, and a second override language until there is a concrete need

Supersedes the docs-only proposal in #688 with executable types, enforcement, and tests.

@Graffioh
Graffioh force-pushed the codex/backend-plan-boundary branch 5 times, most recently from fd5069f to 1208435 Compare September 2, 2026 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant