Skip to content

docs(server): propose structured inference configuration - #688

Closed
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/inference-config-plan
Closed

docs(server): propose structured inference configuration#688
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:codex/inference-config-plan

Conversation

@Graffioh

@Graffioh Graffioh commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • document the current split between CLI parsing, backend preparation,
    backend construction, environment variables, ServerConfig, and model cards
  • propose BackendArgs -> BackendPlan -> create_backend(plan) as the first
    implementation boundary
  • keep observed runtime facts in a later BackendInfo report and serving
    concerns in ServerConfig
  • stage the argument registry, config-file loader, environment aliases, and
    runtime reporting as follow-up work
  • compare the proposal with current vLLM, SGLang, TensorRT-LLM, TGI, and
    llama.cpp configuration designs

Scope

This is a documentation-only RFC. It does not change runtime behavior or add a
configuration file.

Why this order

The factory currently receives both raw BackendArgs and a partial resolved
plan. server_main can mutate DDTree and SpecLA fields after preparation, so
the factory must re-check the pair. Sealing construction behind one immutable
plan removes that invalid state before Lucebox adds another configuration
source.

Verification

  • based on Luce-Org/lucebox@298031aa
  • checked the described symbols and call sites against current source
  • verified all referenced local paths and external design links
  • git diff --cached --check
  • confirmed balanced Markdown code fences

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant