BlipCode

Minimal, fast, modular Rust harness for LLM inference. Improves existing models with practical inference-time strategies — no weight changes.

• Library + optional CLI, stable Rust, tokio async.
• OpenAI-compatible /chat/completions first (base URL + key via env).
• Streaming is first-class: incremental SSE, deltas without buffering, cancellation stops work, partial output preserved.
• Strategies: baseline (N=1), Self-Consistency (bounded parallel + majority), Self-Refine (critique→revise, keeps original).
• Verifiers: exact, JSON fields, custom fn, external test results, model-judge adapter. Outcomes: pass/fail/inconclusive.
• Budget-aware routing: defaults to single call, escalates only when configured. Cost is estimated, unknown stays unknown.
• Telemetry: latency, TTFT, tokens (when known), calls, retries, verification, termination. JSON to stderr/file, no DB.

Install:

cargo install --path .
export OPENAI_API_KEY=...
blipcode ask "Explain binary search"

More:

blipcode ask "Solve 12*12, just the number" --strategy self-consistency --samples 5
blipcode ask "Review this code" --strategy self-refine --max-iterations 2
blipcode ask "hi" --stream
blipcode bench --dataset ./benchmarks/tasks.jsonl --config ./blipcode.toml --out results.json

Config: copy blipcode.example.toml. No file needed for simplest request. See docs/env.md, docs/capabilities.md, docs/strategies.md, docs/benchmark-method.md, docs/backlog.md.

Refs: Self-Consistency https://arxiv.org/abs/2203.11171, Self-Refine https://arxiv.org/abs/2303.17651.

Status: v0.1. See CHANGELOG.md and docs/benchmark-report.md template. Mock tests reproduce with cargo test; live runs need OPENAI_API_KEY.