Skip to content

feat: make local GGUF setup resumable and restart-safe - #98

Merged
XYAIStudio merged 5 commits into
developfrom
codex/local-runtime-sidecar
Sep 29, 2026
Merged

XYAIStudio merged 5 commits into
developfrom
codex/local-runtime-sidecar

Conversation

@XYAIStudio

@XYAIStudio XYAIStudio commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

FreeOS users can now start from a computer with neither Ollama nor an existing GGUF file. Settings → Models presents a pinned starter catalog, downloads the selected official Qwen GGUF with live progress, verifies its SHA-256, starts the bundled llama.cpp runtime, benchmarks it, and selects it for local chat only after a successful test.

This update closes the reliability gaps needed for the 0.0.7 flow:

  • GGUF download jobs persist under the FreeOS home directory and resume from .part files after cancellation, network failure, or application restart.
  • Concurrent downloads of the same catalog item are deduplicated and disk space is checked before transfer.
  • The user-selected llama.cpp model is restored after FreeOS restarts; an explicit Stop clears that intent.
  • Benchmark results persist across sessions, and interrupted downloads are surfaced with a one-click Resume action.
  • API and Chinese installation documentation cover the catalog, download, llama.cpp, benchmark, and default-model flow.

The catalog currently offers pinned official Apache-2.0 Qwen2.5 1.5B and 3B Q4_K_M builds. The runtime remains pinned and checksum-verified, binds only to 127.0.0.1:11435, uses an argument vector rather than a shell command, logs under the FreeOS home directory, and stops only the process owned by FreeOS. Ollama remains an optional runtime and import path.

Validation:

  • PYTHONUTF8=1 uv run pytest -n 4 -m "not live" — 3588 passed, 114 skipped
  • uv run mypy src/octop — no issues in 590 source files
  • uv run ruff check src tests and uv run ruff format --check src tests — 1158 files formatted
  • focused backend tests — 13 passed
  • focused dashboard tests — 10 passed
  • dashboard ESLint — 0 errors; tsc --noEmit and production build passed
  • GitHub Linux, Windows, and four-platform desktop package jobs rerun on each pushed commit

This PR remains stacked on #97 so the implementation and architecture plan can be reviewed separately. #97 now targets develop under the repository release policy.

@XYAIStudio
XYAIStudio force-pushed the codex/local-runtime-sidecar branch from 6371812 to 2dd4686 Compare September 28, 2026 16:27
@XYAIStudio XYAIStudio changed the title feat: run GGUF models with bundled llama.cpp feat: bootstrap local GGUF models with bundled llama.cpp Sep 28, 2026
@XYAIStudio XYAIStudio changed the title feat: bootstrap local GGUF models with bundled llama.cpp feat: make local GGUF setup resumable and restart-safe Sep 29, 2026
@XYAIStudio
XYAIStudio changed the base branch from codex/local-model-auto-routing-plan to develop September 29, 2026 07:57
@XYAIStudio
XYAIStudio merged commit 8ba2fd6 into develop Sep 29, 2026
9 checks passed
@XYAIStudio XYAIStudio mentioned this pull request Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant