Docs lag shipped code by about six weeks. Concrete items:
README points at files that do not exist:
README.md:157 — python examples/quickstart.py; no such file (examples/ has evaluate.py, shadow_from_traces.py, and per-backend method examples only).
README.md:162 — Colab badge links examples/colab_test_drive.ipynb; no .ipynb exists anywhere in the repo.
README roadmap lists shipped features as future:
README.md:207 "Eval gates" is under the road ahead, but eval.py shipped 2026-06-20 with a CLI command (shadowlm eval), an example, and tests. Zero README mentions.
README.md:209 "Fleet + teams — GPU job queue" is under the road ahead with an unticked checkbox, but the worker/hub tier shipped 2026-07-13 (worker.py, ws.py, shadowlm worker, Machines page, machine tokens, device routing). Zero README mentions.
traces.py (OTLP GenAI ingestion, 555 lines, exported as slm.traces, with example + sample data + 15 tests) has zero README mentions. It is the offline sibling of capture() and arguably the second headline feature.
Misleading install guidance:
README.md:142 and install.sh:9 describe SHADOWLM_EXTRAS=cli as "lighter: just the UI/CLI", but [cli] resolves to [] (pyproject.toml:94), so it installs the identical full stack.
_cli_entry.py:23-25 tells the user to pip install 'shadowlm[cli]' to fix a ModuleNotFoundError; that extra is empty, so the suggested fix is a no-op that cannot resolve the error it is raised for.
Source docstring drift:
models.py:148-149 lists 12 built-in methods; more_plus is missing (registry has 13, methods/__init__.py:40).
models.py:295,305 document backend: as "auto" | "mlx" | "torch"; the actual accepted set adds remote and verl (backends/__init__.py:79). The same omission appears in five CLI help strings (cli.py:211, 305, 323, 369, 496).
Packaging:
numpy is imported directly (more.py:61,82,116,133, more_plus.py:50,169,278) but is not in dependencies (pyproject.toml:41-65); it resolves only transitively through torch/faiss.
bitsandbytes is Linux-only (pyproject.toml:54) while TrainConfig defaults optim="adamw_8bit"; the README fused-optimizer claim (README.md:81,194) silently does not apply on macOS/Windows.
CLAUDE.md (contributor doc) predates the worker tier:
- Still describes
serve.py as "One job at a time — honest reference tier"; it is now also a job router with per-worker inboxes and inference forwarding.
- Missing entirely:
worker.py, ws.py, traces.py, eval.py, bottleneck.py, the Dashboard and Machines pages, the three-mode auth (password|apikey|none) plus hashed machine tokens, and the multi-server pool pick() in remote.py.
Docs lag shipped code by about six weeks. Concrete items:
README points at files that do not exist:
README.md:157—python examples/quickstart.py; no such file (examples/hasevaluate.py,shadow_from_traces.py, and per-backend method examples only).README.md:162— Colab badge linksexamples/colab_test_drive.ipynb; no.ipynbexists anywhere in the repo.README roadmap lists shipped features as future:
README.md:207"Eval gates" is under the road ahead, buteval.pyshipped 2026-06-20 with a CLI command (shadowlm eval), an example, and tests. Zero README mentions.README.md:209"Fleet + teams — GPU job queue" is under the road ahead with an unticked checkbox, but the worker/hub tier shipped 2026-07-13 (worker.py,ws.py,shadowlm worker, Machines page, machine tokens, device routing). Zero README mentions.traces.py(OTLP GenAI ingestion, 555 lines, exported asslm.traces, with example + sample data + 15 tests) has zero README mentions. It is the offline sibling ofcapture()and arguably the second headline feature.Misleading install guidance:
README.md:142andinstall.sh:9describeSHADOWLM_EXTRAS=clias "lighter: just the UI/CLI", but[cli]resolves to[](pyproject.toml:94), so it installs the identical full stack._cli_entry.py:23-25tells the user topip install 'shadowlm[cli]'to fix a ModuleNotFoundError; that extra is empty, so the suggested fix is a no-op that cannot resolve the error it is raised for.Source docstring drift:
models.py:148-149lists 12 built-in methods;more_plusis missing (registry has 13,methods/__init__.py:40).models.py:295,305documentbackend:as"auto" | "mlx" | "torch"; the actual accepted set addsremoteandverl(backends/__init__.py:79). The same omission appears in five CLI help strings (cli.py:211, 305, 323, 369, 496).Packaging:
numpyis imported directly (more.py:61,82,116,133,more_plus.py:50,169,278) but is not independencies(pyproject.toml:41-65); it resolves only transitively through torch/faiss.bitsandbytesis Linux-only (pyproject.toml:54) whileTrainConfigdefaultsoptim="adamw_8bit"; the README fused-optimizer claim (README.md:81,194) silently does not apply on macOS/Windows.CLAUDE.md (contributor doc) predates the worker tier:
serve.pyas "One job at a time — honest reference tier"; it is now also a job router with per-worker inboxes and inference forwarding.worker.py,ws.py,traces.py,eval.py,bottleneck.py, the Dashboard and Machines pages, the three-mode auth (password|apikey|none) plus hashed machine tokens, and the multi-server poolpick()inremote.py.