Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fc0b698
Add MLX as an engine choice with its settings, values and pure rules
senadaruc Oct 14, 2026
1a27a04
Run mlx_lm.server as a managed loopback child process and route MLX s…
senadaruc Oct 14, 2026
94e0ef2
Wire MLX into the app, Settings and the menu bar, and count its serve…
senadaruc Oct 14, 2026
96c3f9e
Document the MLX engine
senadaruc Oct 14, 2026
e20d353
Regenerate the Xcode project with XcodeGen
senadaruc Oct 14, 2026
862999e
Add selection rewrite values, rules and engine
senadaruc Oct 14, 2026
02d7a01
Add the rewrite popup, the wand on the field-edge icon, and Settings …
senadaruc Oct 14, 2026
cdc816a
Document selection rewrite
senadaruc Oct 14, 2026
82e6aa7
Satisfy strict SwiftLint
senadaruc Oct 14, 2026
e368589
Merge feat/mlx-engine-main (#868) into the rewrite branch (#869)
senadaruc Oct 14, 2026
4ca33ad
Give Rewrite a provider per engine that reuses autocomplete's model
senadaruc Oct 14, 2026
5029f76
Offer every engine in Settings -> Rewrite and wire the providers
senadaruc Oct 14, 2026
4615bbc
Document Rewrite on every engine
senadaruc Oct 14, 2026
af5f3ba
Cut the base model's stop text and give Longer room to finish
senadaruc Oct 14, 2026
c4899bb
Check rewrites against the user's own text, by word and by value
senadaruc Oct 14, 2026
3c992bc
Stop a broken MLX server even when Cotabby quits right after the report
senadaruc Oct 14, 2026
82bf9fb
Count an MLX model as downloaded only when its refs/main snapshot is …
senadaruc Oct 14, 2026
4147c3b
Show a custom MLX model's download progress, Cancel and failure
senadaruc Oct 14, 2026
e595546
Reject Hugging Face dataset and Space links as MLX model ids
senadaruc Oct 14, 2026
2039043
Guard rewrite arming, Return and Replace
senadaruc Oct 14, 2026
b816264
Bound the rewrite preview, require Claude's end event, refresh the mo…
senadaruc Oct 14, 2026
76d47a4
Merge feat/selection-rewrite-main (#869) review fixes into the engine…
senadaruc Oct 14, 2026
1a4a0f8
Merge feat/mlx-engine-main (#868) review fixes into the engines branch
senadaruc Oct 14, 2026
f6bf7c3
Cut the base model's stop text on the endpoint and MLX completions paths
senadaruc Oct 14, 2026
38fb85e
Check a popup's whole endpoint URL, not only its host
senadaruc Oct 14, 2026
5a7f0a9
Wait for Rewrite's MLX server to stop before autocomplete starts MLX
senadaruc Oct 14, 2026
f40bdf7
Keep a model-list error under the service it belongs to; name Engine …
senadaruc Oct 14, 2026
0e4a4e7
Release RewritePreferencesStore off the main actor's deinit shim
senadaruc Oct 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,9 +164,11 @@ from Accessibility and Input Monitoring.

Runtime generation is split by responsibility:

- `SuggestionEngineRouter`: selects Apple Intelligence, Open Source, or the configured
- `SuggestionEngineRouter`: selects Apple Intelligence, Open Source, MLX, or the configured
OpenAI-compatible endpoint and owns the narrow Apple-to-llama language fallback.
- `FoundationModelSuggestionEngine`: Apple on-device generation path.
- `MLXServerManager`: Cotabby's `mlx_lm.server` child process (loopback only), with
`MLXToolchainProvisioner` and `MLXModelDownloader` for its install and models.
- `LlamaSuggestionEngine`: request-to-prompt, llama result handling, and cache reset handoff.
- `OpenAICompatibleSuggestionEngine`: completion/chat transport, SSE streaming, and Ollama preload
behavior for the configured endpoint.
Expand Down Expand Up @@ -198,7 +200,9 @@ The Swift generation loop owns the maximum output-token budget.
reads as overwriting them. Change those pure helpers and their tests rather than nudging offsets
in the controller.
- `SuggestionOverlayPresenter` decides whether a suggestion should be shown or hidden.
- `ActivationIndicatorController` owns the optional caret/field-edge indicator.
- `ActivationIndicatorController` owns the optional caret/field-edge indicator. Selection rewrite
borrows it as its wand button while text is selected (`showRewrite` / `endRewrite`);
`SelectionRewriteCoordinator` owns that flow, its popup and its Replace checks.
- `FocusDebugOverlayController` is for developer visibility and should stay gated behind debug
options, not normal user settings.
- Settings panes (under `Cotabby/UI/Settings/Panes/`) and onboarding views should remain
Expand Down Expand Up @@ -256,6 +260,9 @@ Console.app stream.
- `~/Library/Logs/Cotabby/llm-io.jsonl` — full LLM prompts and completions, one record per
generation. Shares `request_id` with the main log so a single suggestion can be joined across
files.
- `~/Library/Logs/<app>/mlx-server.log` — everything Cotabby's `mlx_lm.server` wrote to stderr,
one header line (pid, model, port) per server start; rotated to `mlx-server.log.1` at 2 MB.
Without the flag only the last lines stay in memory for the MLX failure message.
- `~/Desktop/cotabby-ax-dump.txt` — most recent Chrome AX tree snapshot. Overwritten on each
Chrome focus change (debounced by focused-element identity).
- Rotated previous logs: `*.jsonl.1` (one-step rotation when a file exceeds 10 MB).
Expand Down Expand Up @@ -295,6 +302,7 @@ jq 'select(.category == "runtime")' ~/Library/Logs/Cotabby/cotabby.jsonl
- Wrong text inserted → look up the request in `llm-io.jsonl`, then walk `suggestion` for
acceptance
- Model won't load / decode fails → `runtime` + `models`
- MLX failed or keeps restarting → `runtime`, then `mlx-server.log`
- Permission dialog loop → `app` (permission state transitions)
- Chrome-specific weirdness → start with `~/Desktop/cotabby-ax-dump.txt`, then `focus`
- Wrong backend chosen → `suggestion` router selection log (`engine`, `fallback_engine`)
Expand Down
39 changes: 38 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ CGEvent and focus poll
-> SuggestionRequestFactory
-> bounded AX / surface / clipboard / visual / user context
-> SuggestionEngineRouter
-> Apple | in-process llama | OpenAI-compatible endpoint
-> Apple | in-process llama | MLX (loopback mlx_lm.server) | OpenAI-compatible endpoint
-> SuggestionTextNormalizer + seam guards
-> SuggestionInteractionState
-> SuggestionOverlayPresenter -> OverlayController
Expand Down Expand Up @@ -257,6 +257,12 @@ text. Posting events is not treated as proof of success; the later AX snapshot i
for completion/chat APIs and SSE streams. The default is loopback Ollama at
http://127.0.0.1:11434/v1; LAN and public HTTPS endpoints are supported, while insecure public HTTP
is rejected.
- MLX: `OpenAICompatibleSuggestionEngine.mlx` talks to an `mlx_lm.server` child process that
[MLXServerManager.swift](Cotabby/Services/Runtime/MLX/MLXServerManager.swift) runs on 127.0.0.1.
`MLXToolchainProvisioner` finds mlx-lm or installs it with a checksum- and signature-pinned uv
under Application Support; `MLXModelDownloader` manages MLX models in the Hugging Face cache;
`MLXServerDesireResolver` decides when the server runs. MLX is on-device (typing history
allowed) but takes no speculative work.

LlamaRuntimeCore is a nonisolated lock/condition-protected native boundary, not a Swift actor. Its
autocomplete lock serializes cache/decode state, and its lifecycle condition prevents shutdown from
Expand Down Expand Up @@ -498,6 +504,36 @@ the single input-capture slot between:
Both use pure trigger state machines, stay pinned to one supported focus sequence, and cancel on
focus change or incompatible input. They do not call a language model.

Selection rewrite is a sibling feature with its own state machine,
[SelectionRewriteCoordinator.swift](Cotabby/App/Coordinators/Rewrite/SelectionRewriteCoordinator.swift)
(hidden → button → popup → generating → preview). Focus snapshots carry the selected text and its
range in the host's value (`selectedText`, `hostSelection`). A stable selection that
[SelectionRewriteEligibility.swift](Cotabby/Support/Rewrite/SelectionRewriteEligibility.swift) accepts
(two words or more, not secure, not a terminal, Cotabby on for the app) turns the field-edge icon into a
wand (`ActivationIndicatorController.showRewrite`) and arms ⌃⌥R through a `TemporaryKeyTap` that exists only
while that selection is armed; the "Rewrite Selected Text" switch in Settings → Rewrite turns all of it off. The popup is
a non-activating panel that takes the keyboard while open, so Return can never reach a chat app.
[RewriteEngine.swift](Cotabby/Services/Rewrite/RewriteEngine.swift) runs one rewrite on the provider
chosen in Settings → Rewrite. The default, "Same as autocomplete", is resolved by the pure
`RewriteProviderResolver` to the provider of the current autocomplete engine, except that it never
sends text off the Mac on its own: an autocomplete endpoint on a LAN or public host resolves to Apple
Intelligence, else Open Source (choosing Local Endpoint explicitly still uses that host). Every local provider
reuses autocomplete's model; there is no rewrite-only model: `AppleIntelligenceRewriteProvider`,
`OpenSourceRewriteProvider` (the selected llama base model with an example-style prompt that stops at
the next `Draft:`), `LocalEndpointRewriteProvider` (autocomplete's OpenAI-compatible endpoint: chat
mode gets the instruct prompt, completions mode the base prompt with the standard `stop` field), and
`MLXRewriteProvider` (autocomplete's MLX server when autocomplete is on MLX, never a second one;
otherwise an on-demand server with autocomplete's MLX model, owned by `MLXRewriteServerController`,
with its own pid file, stopped after five idle minutes). `APIRewriteProvider` covers Claude via
Anthropic's Messages API and other services via OpenAI-compatible chat completions, each service's key
in its own Keychain item. The popup names the host whenever the resolved provider sends the text off
this Mac, including an explicitly chosen Local Endpoint on a LAN or public host. [RewritePromptBuilder.swift](Cotabby/Support/Rewrite/RewritePromptBuilder.swift)
names the draft's language (`RewriteLanguageDetector`), `RewriteOutputCleaner` strips prefaces, fences
and quotes and rejects runaway output, and `RewriteSpecificsCheck` flags numbers and names the user did
not write, which turns plain Replace off. Replace pastes over the selection only after
[SelectionReplaceValidator.swift](Cotabby/Support/Rewrite/SelectionReplaceValidator.swift) confirms the
same field, text and range, then reads the field back; anything else goes to the clipboard with a notice.

[SettingsCoordinator.swift](Cotabby/App/Coordinators/SettingsCoordinator.swift) and
[WelcomeCoordinator.swift](Cotabby/App/Coordinators/WelcomeCoordinator.swift) own app-lifetime AppKit
windows hosting SwiftUI content. Settings and onboarding observe the shared graph. Hiding the menu bar
Expand Down Expand Up @@ -583,6 +619,7 @@ policies, prompt utilities, normalization, and layout logic should receive focus
| Clipboard context is irrelevant | ClipboardRelevanceFilter, ClipboardContentDistiller |
| Screenshot context is stale/noisy | VisualContextCoordinator, OCRTextHygiene |
| Emoji or macro conflicts with suggestions | InlineCommandCoordinator, feature trigger machine |
| Rewrite wand missing, popup misplaced, or Replace copies instead | SelectionRewriteCoordinator, SelectionRewriteEligibility, SelectionReplaceValidator |
| Permission loop or lost grant | PermissionManager, PermissionGuidanceController, app identity |
| Settings/onboarding window issue | SettingsCoordinator, WelcomeCoordinator |
| Settings persistence or domain mismatch | SuggestionSettingsModel, SuggestionSettingsData, SuggestionSettingsStore |
Expand Down
Loading
Loading