diff --git a/AGENTS.md b/AGENTS.md index 1cb70e52f..a4c56d2ad 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -164,9 +164,11 @@ from Accessibility and Input Monitoring. Runtime generation is split by responsibility: -- `SuggestionEngineRouter`: selects Apple Intelligence, Open Source, or the configured +- `SuggestionEngineRouter`: selects Apple Intelligence, Open Source, MLX, or the configured OpenAI-compatible endpoint and owns the narrow Apple-to-llama language fallback. - `FoundationModelSuggestionEngine`: Apple on-device generation path. +- `MLXServerManager`: Cotabby's `mlx_lm.server` child process (loopback only), with + `MLXToolchainProvisioner` and `MLXModelDownloader` for its install and models. - `LlamaSuggestionEngine`: request-to-prompt, llama result handling, and cache reset handoff. - `OpenAICompatibleSuggestionEngine`: completion/chat transport, SSE streaming, and Ollama preload behavior for the configured endpoint. @@ -256,6 +258,9 @@ Console.app stream. - `~/Library/Logs/Cotabby/llm-io.jsonl` — full LLM prompts and completions, one record per generation. Shares `request_id` with the main log so a single suggestion can be joined across files. +- `~/Library/Logs//mlx-server.log` — everything Cotabby's `mlx_lm.server` wrote to stderr, + one header line (pid, model, port) per server start; rotated to `mlx-server.log.1` at 2 MB. + Without the flag only the last lines stay in memory for the MLX failure message. - `~/Desktop/cotabby-ax-dump.txt` — most recent Chrome AX tree snapshot. Overwritten on each Chrome focus change (debounced by focused-element identity). - Rotated previous logs: `*.jsonl.1` (one-step rotation when a file exceeds 10 MB). @@ -295,6 +300,7 @@ jq 'select(.category == "runtime")' ~/Library/Logs/Cotabby/cotabby.jsonl - Wrong text inserted → look up the request in `llm-io.jsonl`, then walk `suggestion` for acceptance - Model won't load / decode fails → `runtime` + `models` +- MLX failed or keeps restarting → `runtime`, then `mlx-server.log` - Permission dialog loop → `app` (permission state transitions) - Chrome-specific weirdness → start with `~/Desktop/cotabby-ax-dump.txt`, then `focus` - Wrong backend chosen → `suggestion` router selection log (`engine`, `fallback_engine`) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 52278b986..1cead7388 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -78,7 +78,7 @@ CGEvent and focus poll -> SuggestionRequestFactory -> bounded AX / surface / clipboard / visual / user context -> SuggestionEngineRouter - -> Apple | in-process llama | OpenAI-compatible endpoint + -> Apple | in-process llama | MLX (loopback mlx_lm.server) | OpenAI-compatible endpoint -> SuggestionTextNormalizer + seam guards -> SuggestionInteractionState -> SuggestionOverlayPresenter -> OverlayController @@ -257,6 +257,12 @@ text. Posting events is not treated as proof of success; the later AX snapshot i for completion/chat APIs and SSE streams. The default is loopback Ollama at http://127.0.0.1:11434/v1; LAN and public HTTPS endpoints are supported, while insecure public HTTP is rejected. +- MLX: `OpenAICompatibleSuggestionEngine.mlx` talks to an `mlx_lm.server` child process that + [MLXServerManager.swift](Cotabby/Services/Runtime/MLX/MLXServerManager.swift) runs on 127.0.0.1. + `MLXToolchainProvisioner` finds mlx-lm or installs it with a checksum- and signature-pinned uv + under Application Support; `MLXModelDownloader` manages MLX models in the Hugging Face cache; + `MLXServerDesireResolver` decides when the server runs. MLX is on-device (typing history + allowed) but takes no speculative work. LlamaRuntimeCore is a nonisolated lock/condition-protected native boundary, not a Swift actor. Its autocomplete lock serializes cache/decode state, and its lifecycle condition prevents shutdown from diff --git a/Cotabby.xcodeproj/project.pbxproj b/Cotabby.xcodeproj/project.pbxproj index efe7ecb17..f341990ce 100644 --- a/Cotabby.xcodeproj/project.pbxproj +++ b/Cotabby.xcodeproj/project.pbxproj @@ -36,6 +36,7 @@ 05CFA72F4FF44A0CDBEFED3C /* PostExhaustionAcceptanceState.swift in Sources */ = {isa = PBXBuildFile; fileRef = 22FA75F9AD318B9084B23ECF /* PostExhaustionAcceptanceState.swift */; }; 067371AD898788392F158EDE /* PhrasePredictionScreenContext.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6E6A9735EBD163A2E8AD67A5 /* PhrasePredictionScreenContext.swift */; }; 07639673B5A52D2075A9ED57 /* VisualContextModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4A51B1559F4CE6728D39C959 /* VisualContextModels.swift */; }; + 0787F247FDEE0287593770D7 /* MLXProcessRunnerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = AB4A82DFAD71F7568E801FB8 /* MLXProcessRunnerTests.swift */; }; 07E54011F501E14AC15DE2B7 /* OnboardingTemplateFeatureList.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7174EF88BA41ACC0A74CA582 /* OnboardingTemplateFeatureList.swift */; }; 081EB4DF65AD5736DE7FBFDE /* MirrorOverlayLayoutTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2D962AC9D24D00957562CA7B /* MirrorOverlayLayoutTests.swift */; }; 085BB87581DFFA260A630E24 /* CotabbyBrand.swift in Sources */ = {isa = PBXBuildFile; fileRef = D3C1C9A9BF1F74C16E4401EC /* CotabbyBrand.swift */; }; @@ -44,6 +45,7 @@ 0901FA0B0FF3593645189449 /* WelcomePersonalizeStepView.swift in Sources */ = {isa = PBXBuildFile; fileRef = F26F50467C83B202FB4BAC2A /* WelcomePersonalizeStepView.swift */; }; 09112D213315AA1C70E303CE /* SuggestionCoordinator+Continuation.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3FFC1D1322005305C7D482D2 /* SuggestionCoordinator+Continuation.swift */; }; 094DE1CA4A821D722F173F9D /* SuggestionDebugLogger.swift in Sources */ = {isa = PBXBuildFile; fileRef = 003594B09C83EF2DF35577D5 /* SuggestionDebugLogger.swift */; }; + 098F4520E2D5BEA2CE45C4A1 /* ResourceUsageSummaryTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7E348E1CB10B4EAAC9CA1572 /* ResourceUsageSummaryTests.swift */; }; 09B092E8A682D127FC9872A7 /* HuggingFaceAPIClient.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4D6D4D7CAD20C94DF9EF21D2 /* HuggingFaceAPIClient.swift */; }; 0A08A88B773845EF9A27DF0F /* SymSpellCorrector.swift in Sources */ = {isa = PBXBuildFile; fileRef = 12E1E72C09460390583D98D1 /* SymSpellCorrector.swift */; }; 0A15FAD03A93D57372D267E0 /* VisualContextCoordinator.swift in Sources */ = {isa = PBXBuildFile; fileRef = A854CAFB1F557BC4CAED8819 /* VisualContextCoordinator.swift */; }; @@ -63,14 +65,17 @@ 0C8D3E1DC9D7B0FC7664055A /* GhostTextColorPreset.swift in Sources */ = {isa = PBXBuildFile; fileRef = 74AB26FDFBBAC0C6375C4192 /* GhostTextColorPreset.swift */; }; 0CF057842A15F79FB3C69462 /* ModelDownloadManager.swift in Sources */ = {isa = PBXBuildFile; fileRef = 050DC9D858A124FD1CFA4D99 /* ModelDownloadManager.swift */; }; 0CF6C912A1B25718CE090195 /* FocusSnapshotResolver.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6BEFDDFF24B78CDED4EB4206 /* FocusSnapshotResolver.swift */; }; + 0D110609FA275045E66406A5 /* EngineAndModelPaneView+MLX.swift in Sources */ = {isa = PBXBuildFile; fileRef = 057A4153C112521EDCFD1CC2 /* EngineAndModelPaneView+MLX.swift */; }; 0D2CAE170E355D2729FDD9C4 /* SettingsSearchRanker.swift in Sources */ = {isa = PBXBuildFile; fileRef = 57B29DAC600626783649475A /* SettingsSearchRanker.swift */; }; 0DA822FDD75309CA14FCB915 /* WelcomeView.swift in Sources */ = {isa = PBXBuildFile; fileRef = A9805B5D25A6E261D92E0ADD /* WelcomeView.swift */; }; 0DBDD71AB17F87D6558DC07D /* VisualContextExcerptSelector.swift in Sources */ = {isa = PBXBuildFile; fileRef = 47457FED99065966328047EB /* VisualContextExcerptSelector.swift */; }; 0DE687D08BE8AA33F0A91787 /* TokenCountEstimatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9AA97052545B05BA82450A6D /* TokenCountEstimatorTests.swift */; }; 0E6692D2A7AEEDF72FCF1AE2 /* ExtendedContextTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = DA0447A27B293394D9379C48 /* ExtendedContextTests.swift */; }; + 0ED3FD222E15951E0FCAC9ED /* MLXProcessRunner.swift in Sources */ = {isa = PBXBuildFile; fileRef = DECAC67D186C067A6F75517D /* MLXProcessRunner.swift */; }; 1010A070B4E0F0A2FDF45112 /* CompletionContentPolicyTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BDA5F007F5185BD597550607 /* CompletionContentPolicyTests.swift */; }; 102E1EDC04636AFB607A8249 /* ActiveSuggestionSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = 04F95814016D80E0A23974F0 /* ActiveSuggestionSession.swift */; }; 105187F185A579BC67CA2A93 /* AXHelper.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5B5D1A7D938F633C6EE094F7 /* AXHelper.swift */; }; + 1064D280B1851263DA3FE745 /* MenuResourceMonitor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1BC7B3FAE8ABAFE9910AEED2 /* MenuResourceMonitor.swift */; }; 1092E77D9D92DAC4808ABF9A /* GhostFontResolver.swift in Sources */ = {isa = PBXBuildFile; fileRef = F3BB26526298A85454330591 /* GhostFontResolver.swift */; }; 109F9BD7E1AE3E7FB0C712AA /* GhostTextLayoutTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3EB202C5AD6079174A14330C /* GhostTextLayoutTests.swift */; }; 1104ABD01283774EDF249EF0 /* LLMIOFileHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = 56A2F75C66BECA61A722D3FB /* LLMIOFileHandler.swift */; }; @@ -101,6 +106,7 @@ 16BD2E6ECD6A818711A799D3 /* InsertedTextAdvanceTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 35AA2C8F42B510F013D86C3C /* InsertedTextAdvanceTests.swift */; }; 16D638B016CEF136AB6BCC1C /* SuggestionPhraseAcceptanceTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2D520FBB6AE0E786FFAD1B11 /* SuggestionPhraseAcceptanceTests.swift */; }; 16E1A22AA7F4827E69C7F2CE /* SuggestionSettingsStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8AB17696CE79982039061F29 /* SuggestionSettingsStore.swift */; }; + 1727CCFF2E46A8E471AE78B9 /* MLXEngineWiringTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 36C581E9E992813892959DBE /* MLXEngineWiringTests.swift */; }; 1741F4AFFEA90CD6F81959F3 /* ArithmeticEvaluatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = D381F1752705DC80E491B025 /* ArithmeticEvaluatorTests.swift */; }; 1816D3D625F561209590D45D /* TagChip.swift in Sources */ = {isa = PBXBuildFile; fileRef = A04DFC5BD7821D73DA21D7DB /* TagChip.swift */; }; 181DB7463BA225142F02DEFD /* InsertionStrategySelectorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 12BB15F7F2E595129B190E89 /* InsertionStrategySelectorTests.swift */; }; @@ -117,6 +123,7 @@ 19EB4D1C2D18C0ADC70BAD29 /* EngineAndModelPaneView+OpenSource.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8B2C3A2CEC6E0A13E2102D15 /* EngineAndModelPaneView+OpenSource.swift */; }; 1A0EEE893EA683AE79EA76C3 /* WordPrefixIndex.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6428AF104998876A60AC602C /* WordPrefixIndex.swift */; }; 1A4A2FCA8F08258A8A70A057 /* LGPL-3.0.txt in Resources */ = {isa = PBXBuildFile; fileRef = 7513810E78F3C94FE972EB07 /* LGPL-3.0.txt */; }; + 1A789303EB793105611E31AF /* MLXServerDesireResolverTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 27049BD6A6B9E46DE07E01C8 /* MLXServerDesireResolverTests.swift */; }; 1A9FB5C832377E951C572606 /* HostBaselineCalibratorTypefaceTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 147D8EAD035A66FCA1D07E37 /* HostBaselineCalibratorTypefaceTests.swift */; }; 1AAC3A35721942F618E5C37F /* ApplicationBundleMetadata.swift in Sources */ = {isa = PBXBuildFile; fileRef = BD1E28CF46BF59ABDC3056BF /* ApplicationBundleMetadata.swift */; }; 1AB037C6DC95769F4765B5CA /* SettingsAttentionEvaluator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 79AAAB1C46A0E5A9C6FC7D3D /* SettingsAttentionEvaluator.swift */; }; @@ -177,6 +184,7 @@ 29D8E4E0F4D4ADF4136E20AE /* EmojiPopularity.swift in Sources */ = {isa = PBXBuildFile; fileRef = 77DB52780A112A686C06D7E3 /* EmojiPopularity.swift */; }; 29E20FF0F413E723D2A526C4 /* DecodeStopPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = C319B4FF652512A69377DD5C /* DecodeStopPolicy.swift */; }; 2A3FC700422694CAF70C5449 /* SuggestionResult.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8543EBA09E11A5066880CC91 /* SuggestionResult.swift */; }; + 2A652B3FF4213C2454E1814B /* MLXServerLogFilter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 044F1B6638860E6E8C37B5C6 /* MLXServerLogFilter.swift */; }; 2AA1A89BD542ADFCE16F594C /* SuggestionAnchorCache.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4861CC2533F3EA222A0833A9 /* SuggestionAnchorCache.swift */; }; 2AF12199E0FD9B802BF35A10 /* SpellingDictionaryCatalog.swift in Sources */ = {isa = PBXBuildFile; fileRef = 25D41CC179EC04093971A0D0 /* SpellingDictionaryCatalog.swift */; }; 2B355C61C20866199F414EB9 /* PermissionOverlayTracker.swift in Sources */ = {isa = PBXBuildFile; fileRef = E286B9A912808088FDA6B4C4 /* PermissionOverlayTracker.swift */; }; @@ -212,6 +220,8 @@ 332F2AE6277FB6E871589E4D /* MacroControllerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5C64BFD7273D1B637AD880AA /* MacroControllerTests.swift */; }; 3368DFBAA36527B7BB09EA67 /* ApplicationBundleMetadataTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8C151BF4D39485E5CFACFECB /* ApplicationBundleMetadataTests.swift */; }; 33EC776945F8CE612A8E85A3 /* CustomRulesEditor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9E7D1184F0BC707F0A9148DD /* CustomRulesEditor.swift */; }; + 34D97C80628E0085F151EFE7 /* MLXGeneratorFailure.swift in Sources */ = {isa = PBXBuildFile; fileRef = 311D0D0F37A113911CAC1D07 /* MLXGeneratorFailure.swift */; }; + 351264BEFC3D30B585F480FD /* MLXToolchainProvisioner.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9CFBBF67EDCC6D7E19FFD04C /* MLXToolchainProvisioner.swift */; }; 35173B2E2394D942043FB086 /* CompletionContentPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 384D85534F3C2F40E2FD8781 /* CompletionContentPolicy.swift */; }; 35F1681148A324CBEC797BAF /* MenuBarView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7DE038A6DF4A16B3EB5CEBD3 /* MenuBarView.swift */; }; 3615284684EBC0F0E8843BE6 /* MailHeaderFieldDetector.swift in Sources */ = {isa = PBXBuildFile; fileRef = 47F1A6BCCA20EBE7314B79D3 /* MailHeaderFieldDetector.swift */; }; @@ -258,12 +268,14 @@ 3D10470F0DFB2FEF6A31FC9F /* StreamedGhostTextPolicyTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E35DBAB63F24041B8A30E56A /* StreamedGhostTextPolicyTests.swift */; }; 3D556DDB4C619137308F21F9 /* SettingsIconTile.swift in Sources */ = {isa = PBXBuildFile; fileRef = DB6A4CC16A6970E3E7086B5C /* SettingsIconTile.swift */; }; 3D62542845E1048A93A48749 /* CaretGeometrySelector.swift in Sources */ = {isa = PBXBuildFile; fileRef = B9F5D16600EE9D4B76D04A26 /* CaretGeometrySelector.swift */; }; + 3DBD5276426E05403FDCEB67 /* MLXServerLogWriter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 578A470C822BB93EEC442960 /* MLXServerLogWriter.swift */; }; 3DC75D1C5EF3AB83E3BC5750 /* SuggestionSubsystemContracts.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5652FFF8E718A17D98F87CD0 /* SuggestionSubsystemContracts.swift */; }; 3E1DBB2ABCB2404B59173534 /* SettingsIndex.swift in Sources */ = {isa = PBXBuildFile; fileRef = 34EF2406E7A384F3325AAF9A /* SettingsIndex.swift */; }; 3E235B9AECC972973A94B678 /* ConfidenceSuppressionPolicyTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E2F60BE5EDABE8882CA2F63B /* ConfidenceSuppressionPolicyTests.swift */; }; 3E6BD683DA505C9737463890 /* SettingsNavigationModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1D1AC5EC06664F49A6AE2B17 /* SettingsNavigationModel.swift */; }; 3E78D03ABA7141D344AB8285 /* he.txt in Resources */ = {isa = PBXBuildFile; fileRef = C9C000E46A1E404932F89C81 /* he.txt */; }; 3EAFEEC83B25B023C606A726 /* ContextBuffer.swift in Sources */ = {isa = PBXBuildFile; fileRef = C7B8727111208CB506C25C4F /* ContextBuffer.swift */; }; + 3F7F2CB339DB6B1628A4E861 /* MLXToolchainRulesTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8B87FBEAF425DEE74F008EE5 /* MLXToolchainRulesTests.swift */; }; 3F8E5F610B3AD803DDA007CC /* KeycapView.swift in Sources */ = {isa = PBXBuildFile; fileRef = CF96999230079A768BEFD0FC /* KeycapView.swift */; }; 3FA7429EE4BDE4CE0D8E7DA0 /* SuggestionQualityMetricsStoreTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CC5CD24D2FF35FD36B8D78DE /* SuggestionQualityMetricsStoreTests.swift */; }; 4002A961699B7565A9DB5BF7 /* TypingHistoryPhrasePredictor.swift in Sources */ = {isa = PBXBuildFile; fileRef = F9CE9C2EE56728151E6CE5C7 /* TypingHistoryPhrasePredictor.swift */; }; @@ -278,6 +290,7 @@ 41ACF0EDFE4E69328E4EA292 /* Aria2Locator.swift in Sources */ = {isa = PBXBuildFile; fileRef = A29EA4138BBF767BA8D27223 /* Aria2Locator.swift */; }; 41F36D158005A69CE87921FA /* PermissionOverlayTrackerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = AD003D4EBE530DC0E2B87C24 /* PermissionOverlayTrackerTests.swift */; }; 430EB30535A955C3904C2AD9 /* PerDomainDisableSettings.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6F49C55FFF4352A6763BAE1A /* PerDomainDisableSettings.swift */; }; + 432853D9888B652846A81D8F /* MLXCacheScanner.swift in Sources */ = {isa = PBXBuildFile; fileRef = E83059EDBC917FDE4F7AFE44 /* MLXCacheScanner.swift */; }; 44265797C6EC7798A5E61EF3 /* DateMacroEvaluator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 94A26D86030FC5FA67371C52 /* DateMacroEvaluator.swift */; }; 442999B123DFBAD112231987 /* SuggestionLengthPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = DDA2EA95C425C3EAF246C66D /* SuggestionLengthPolicy.swift */; }; 44B7F10676606D11D0B1D5D0 /* SuggestionCoordinator+Acceptance.swift in Sources */ = {isa = PBXBuildFile; fileRef = CA85693671D2A705F544B566 /* SuggestionCoordinator+Acceptance.swift */; }; @@ -334,6 +347,7 @@ 53F946FF09038C1C751C6E53 /* FoundationModelPromptRenderer.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3A051152D3C503A2339BF0FA /* FoundationModelPromptRenderer.swift */; }; 54091F97C3F64ABB737DC76B /* LanguageCatalog.swift in Sources */ = {isa = PBXBuildFile; fileRef = BE6EA89178F52F6D5B90CBFE /* LanguageCatalog.swift */; }; 540C1F9A70A5C7DB988FBDD4 /* VisualContextExcerptSelector.swift in Sources */ = {isa = PBXBuildFile; fileRef = 47457FED99065966328047EB /* VisualContextExcerptSelector.swift */; }; + 5424621834130DA5BAC8B432 /* MLXToolchainInstaller.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9093E08339428C7A0107D993 /* MLXToolchainInstaller.swift */; }; 54290B1569A7568809014A23 /* SystemSettingsWindowLocator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 177B69917649DAD33BC90388 /* SystemSettingsWindowLocator.swift */; }; 543B45F8FF4A6AFCB9286521 /* Sparkle-LICENSE.txt in Resources */ = {isa = PBXBuildFile; fileRef = 70C489EC7EA50C2B103A6B09 /* Sparkle-LICENSE.txt */; }; 5461DD0133FE632B2EB4936A /* BrowserAppDetectorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 589029534F91D2BE26A65263 /* BrowserAppDetectorTests.swift */; }; @@ -350,6 +364,7 @@ 56987393D95962CD49B4A4E8 /* TypingHistoryPhraseEngine.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0DB8BFC8B80A47088FDDFC24 /* TypingHistoryPhraseEngine.swift */; }; 56D2FDE72FE21C35E8BDD06C /* ActivationIndicatorController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2E379AF08CDFF65D6EFC565E /* ActivationIndicatorController.swift */; }; 56D7E792922B83393A578E5A /* CompletionRenderModePolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 705E9119B1661BF464496D39 /* CompletionRenderModePolicy.swift */; }; + 56EEAD59C2764ACDDFA3E597 /* MLXModelsTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 803C81D876465AAAD03691AB /* MLXModelsTests.swift */; }; 5732858F8FF26E50AE4BFF8C /* DebugForcedSuggestionEngine.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2C8D50115291212D03414CBE /* DebugForcedSuggestionEngine.swift */; }; 57857C599039465ADB0EC3AD /* GhostCaretRefinement.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9A39920BD5E23E4D2E9AF037 /* GhostCaretRefinement.swift */; }; 57BCDFE786675C9793F3E08E /* ControlTokenMarkersTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3D6234872B84ECB56A707945 /* ControlTokenMarkersTests.swift */; }; @@ -372,6 +387,7 @@ 5CBE09685D85DFED89B79437 /* FocusPollBackoffTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = C1337F5C94FA9A6388841805 /* FocusPollBackoffTests.swift */; }; 5CF34E01F13FB242817057E2 /* Sparkle in Frameworks */ = {isa = PBXBuildFile; productRef = 6F134BAB5BC7015B9B2354AF /* Sparkle */; }; 5D26377BB1EADFC3742D6800 /* OpenAICompatibleAPIClient.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1EE99DFA6D7D4B925AFAD1B4 /* OpenAICompatibleAPIClient.swift */; }; + 5D5546A84BF1660CB2CFE9ED /* MLXProcessRunner.swift in Sources */ = {isa = PBXBuildFile; fileRef = DECAC67D186C067A6F75517D /* MLXProcessRunner.swift */; }; 5D6BF62973E33472E20E82B5 /* SuggestionCoordinator+Lifecycle.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7814C85DA992C8124474FD81 /* SuggestionCoordinator+Lifecycle.swift */; }; 5D888AF36EE5B86B99F99987 /* ModelDownloadTransferControls.swift in Sources */ = {isa = PBXBuildFile; fileRef = A25B3A7C488A6D863FE56113 /* ModelDownloadTransferControls.swift */; }; 5E1CE9C9062D2DEB4B96FD5C /* PerDomainDisableSettings.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6F49C55FFF4352A6763BAE1A /* PerDomainDisableSettings.swift */; }; @@ -480,10 +496,12 @@ 753DC144B9394A35A3F395DA /* SettingsSidebarView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BADB38D0160B47637572FC5E /* SettingsSidebarView.swift */; }; 75432BB598E96902A799AA10 /* OpenAICompatibleCredentialStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2B23950F38B1B3C8A5CC5BE5 /* OpenAICompatibleCredentialStore.swift */; }; 757696BA01AE112E3B1241EC /* CaretLagPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = B40016262B83C6C55B6593C7 /* CaretLagPolicy.swift */; }; + 75845759F2066E0558AFDFC7 /* SuggestionEngineKindTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7B70ED445F6B46B9FE4176A5 /* SuggestionEngineKindTests.swift */; }; 764F157C87F3155FBE698A3A /* SuggestionConversationIsolationTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 022734DBBC384802C7CDB97D /* SuggestionConversationIsolationTests.swift */; }; 765A4E8212962910AA3FBA87 /* InputSuppressionControllerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3DBD75CAF6CAC849D5ED9FA4 /* InputSuppressionControllerTests.swift */; }; 76933444A4EF1F1CE5039E9E /* ContextLivePreviewField.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8368940C6C15CC3F932B8DFC /* ContextLivePreviewField.swift */; }; 76B2F9D2C93958F10F65D26F /* SuggestionSessionReconciliationTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9EEDCE7B9BC5B22080B3A8DF /* SuggestionSessionReconciliationTests.swift */; }; + 76C901B3BB16E0BEBC741C43 /* MenuResourceMonitor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1BC7B3FAE8ABAFE9910AEED2 /* MenuResourceMonitor.swift */; }; 76D834C8C72F8AFD737D8433 /* TerminalAppDetectorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 52C557203A5A5F08E68BB11B /* TerminalAppDetectorTests.swift */; }; 77823BCFB29F63615C514927 /* CalendarAccessibilityCaptureGuard.swift in Sources */ = {isa = PBXBuildFile; fileRef = 049102F0ED85618106A042C4 /* CalendarAccessibilityCaptureGuard.swift */; }; 778A81527D05314033B7521C /* SuggestionConfiguration.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6E3BA84BBFD8B2501767516C /* SuggestionConfiguration.swift */; }; @@ -505,6 +523,7 @@ 7C56674EF8DA8869D5116EE6 /* GhostSpaceBoundaryTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1FD9262617968D8D1EF1EA45 /* GhostSpaceBoundaryTests.swift */; }; 7C5BD8FDACC491EF62665FB2 /* TerminalAppDetector.swift in Sources */ = {isa = PBXBuildFile; fileRef = E6DC2CCFDB2BF1F1E1375620 /* TerminalAppDetector.swift */; }; 7CD9B73CE933F15B490B3605 /* SettingsSearchResultRow.swift in Sources */ = {isa = PBXBuildFile; fileRef = D0DB1F528070F8F9108720DB /* SettingsSearchResultRow.swift */; }; + 7D420A402011BE5BB935700C /* MLXServerManagerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2289DB842350106360F00DB9 /* MLXServerManagerTests.swift */; }; 7D4D95E3626A1263DF8A0AC4 /* SuggestionEngineSelectionPolicyTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = F1B95D6456663C72DE3A6AF1 /* SuggestionEngineSelectionPolicyTests.swift */; }; 7D87C0AEF85FA0653C9C6031 /* PerformanceMetricsStoreTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BA991392A72658B758551BBA /* PerformanceMetricsStoreTests.swift */; }; 7DE4F91CADF4E65BCCF4DB17 /* SettingsRowLabel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 44768629365FAD1A66DA3A51 /* SettingsRowLabel.swift */; }; @@ -535,6 +554,7 @@ 84C7C981F9243D1DB1BC90A4 /* PermissionOverlayWindowController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4213A267AE55FE56A071646F /* PermissionOverlayWindowController.swift */; }; 84E9E1F4D55CEB8A9692734E /* MenuBarPresentationObserver.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9DE86A49339BB507D197629D /* MenuBarPresentationObserver.swift */; }; 85178E15A85EB772554220C6 /* WebContentFieldDetector.swift in Sources */ = {isa = PBXBuildFile; fileRef = 178C5C54E1CC7E906DEA1D51 /* WebContentFieldDetector.swift */; }; + 851B43F86320BEBAF8B45EDC /* ResourceUsageSummary.swift in Sources */ = {isa = PBXBuildFile; fileRef = 110BCAD18AFB2D446E5A607D /* ResourceUsageSummary.swift */; }; 852D882AA3AF54EE73BA5306 /* SpellingDictionaryCatalog.swift in Sources */ = {isa = PBXBuildFile; fileRef = 25D41CC179EC04093971A0D0 /* SpellingDictionaryCatalog.swift */; }; 853C19A148DE6D5A59AA2C87 /* ClipboardRelevanceFilterTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 04364C3242A25358E7C85FD6 /* ClipboardRelevanceFilterTests.swift */; }; 8553CEBF40318B0D6DD36B0C /* ChromiumInlineAutocompleteTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = A166301D0A276214F895A963 /* ChromiumInlineAutocompleteTests.swift */; }; @@ -549,6 +569,7 @@ 888FF8B27F8545F1EE9BBA93 /* OpenAICompatibleSuggestionEngine.swift in Sources */ = {isa = PBXBuildFile; fileRef = B241105A4F9C29DD8C8D4A91 /* OpenAICompatibleSuggestionEngine.swift */; }; 88AAFB21F06EE37A392081E1 /* SuggestionSessionReconciler.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2CADFBB03C10309B9048C8E9 /* SuggestionSessionReconciler.swift */; }; 88B5413CB02F1C53E81DC683 /* HostFaceMemoryTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 623D8B3C927836D7C4002AB8 /* HostFaceMemoryTests.swift */; }; + 88FEE83F61272CCB04D1FC83 /* MLXServerDesireResolver.swift in Sources */ = {isa = PBXBuildFile; fileRef = DF574B2D5A31E00ECE381724 /* MLXServerDesireResolver.swift */; }; 89329024F050602EFBC7CC6B /* FocusTracker.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5C9FDF029F7828CAF3FE8850 /* FocusTracker.swift */; }; 89B0199B226B7B6FCEBBBAC3 /* SuggestionCoordinatorRigTestCase.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2471F967A05EB4C8BF7CD242 /* SuggestionCoordinatorRigTestCase.swift */; }; 8AE7B4DE4650D0DA8F6EFFE7 /* GhostTextLayout.swift in Sources */ = {isa = PBXBuildFile; fileRef = C5C0792BF971B702DE0A095E /* GhostTextLayout.swift */; }; @@ -559,9 +580,12 @@ 8C090BA472D90D9682F329BA /* KeyCodeLabels.swift in Sources */ = {isa = PBXBuildFile; fileRef = EF4D6CEDF032A25EF72D7228 /* KeyCodeLabels.swift */; }; 8C45D41F31882E8C58BAB575 /* SettingsPaneScaffold.swift in Sources */ = {isa = PBXBuildFile; fileRef = BE354770CC59461EE0E976D3 /* SettingsPaneScaffold.swift */; }; 8C81A7D12CDC459F5F79AD9D /* SuggestionCoordinator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9E76C76AEE756F43787EEECE /* SuggestionCoordinator.swift */; }; + 8D04ACCC0BDD08806612E029 /* MLXHealthProbe.swift in Sources */ = {isa = PBXBuildFile; fileRef = E2E6128CA72E5FE4BA856229 /* MLXHealthProbe.swift */; }; + 8D1EB57587E5BB84758C6797 /* MLXServerManager.swift in Sources */ = {isa = PBXBuildFile; fileRef = 33291D41B4D72725738C3562 /* MLXServerManager.swift */; }; 8D380BEC82C2969F3ED2161A /* HardwareCapabilityProbe.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9F63A5EF992329FB9F6C4C26 /* HardwareCapabilityProbe.swift */; }; 8D83B47447409074AAD1A423 /* MidWordContinuationPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = C3F56B79BF030507CC7FAE4D /* MidWordContinuationPolicy.swift */; }; 8DA36F1521B6A59D8C20AC59 /* Logging in Frameworks */ = {isa = PBXBuildFile; productRef = 5A60D1467BBFECB3DFEB39C2 /* Logging */; }; + 8DB6D558C9696ED2D562384B /* MLXServerLogFilter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 044F1B6638860E6E8C37B5C6 /* MLXServerLogFilter.swift */; }; 8DCEEB9FCB3F7C90B7B4917F /* TypingHistoryIndex.swift in Sources */ = {isa = PBXBuildFile; fileRef = DAC3203A94F736B83237FEC9 /* TypingHistoryIndex.swift */; }; 8E209BC819EBBCA0E7F10775 /* FoundationModelAvailabilityService.swift in Sources */ = {isa = PBXBuildFile; fileRef = FA97ED2A10499771F9C32B64 /* FoundationModelAvailabilityService.swift */; }; 8E236047C490CD266E603823 /* TypingHistoryPhrasePredictorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7FB5276B7DDE61030A57EF9A /* TypingHistoryPhrasePredictorTests.swift */; }; @@ -577,6 +601,7 @@ 908D99B538976C07833157E8 /* OnboardingFeatureShowcase.swift in Sources */ = {isa = PBXBuildFile; fileRef = 926B332E7B4CFEE42C4CAA75 /* OnboardingFeatureShowcase.swift */; }; 90D53306BE8369534BE8ED4F /* CaretWordContext.swift in Sources */ = {isa = PBXBuildFile; fileRef = 976E3D6790D2A3594BCEEA23 /* CaretWordContext.swift */; }; 9116AADE58B526162FBF4676 /* DebugForcedSuggestionEngine.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2C8D50115291212D03414CBE /* DebugForcedSuggestionEngine.swift */; }; + 9153232EF26A3A9B10C8E62D /* MLXModelCompatibilityTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CC8B01E696534BACF09ED6C7 /* MLXModelCompatibilityTests.swift */; }; 917E9217724C62C8C8E4E1C9 /* FocusCapabilityFlickerGate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 79AA39C97E8A959E9A5B6F04 /* FocusCapabilityFlickerGate.swift */; }; 91979FC94BCE7A80F74EC47F /* Aria2StagingDirectoryResolver.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAA0ED5F9A319942CA024F93 /* Aria2StagingDirectoryResolver.swift */; }; 920DE073BEE1DC78B73108D0 /* OnboardingFlowSteps.swift in Sources */ = {isa = PBXBuildFile; fileRef = 64AEF94A48DF96D6BEE3048C /* OnboardingFlowSteps.swift */; }; @@ -598,6 +623,7 @@ 948AD3BD25C113539DF12FD7 /* VisualContextModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4A51B1559F4CE6728D39C959 /* VisualContextModels.swift */; }; 949409B5B133BF4960691AE8 /* SuggestionRequestFactory.swift in Sources */ = {isa = PBXBuildFile; fileRef = DD834BCFF812935889E16D5A /* SuggestionRequestFactory.swift */; }; 94A93C9AECB84D56E3347601 /* FocusPollBackoff.swift in Sources */ = {isa = PBXBuildFile; fileRef = ECD5C8648740E7050CCA657F /* FocusPollBackoff.swift */; }; + 94D0A4675205578C85A1561E /* MLXModelCompatibility.swift in Sources */ = {isa = PBXBuildFile; fileRef = C659AC4B22F9E2E0D69D137E /* MLXModelCompatibility.swift */; }; 94FA4632A562E454472BE1C6 /* HostBaselineCalibrator.swift in Sources */ = {isa = PBXBuildFile; fileRef = DDC1ECBE9D1EDE855B57A913 /* HostBaselineCalibrator.swift */; }; 952729C6514FBC40D6E7D1D4 /* CotabbyInference in Frameworks */ = {isa = PBXBuildFile; productRef = D11AB0F962CCC5BD79B630CA /* CotabbyInference */; }; 95B1C40207F57D7E059B9965 /* de.txt in Resources */ = {isa = PBXBuildFile; fileRef = C648EBB10D7F8E0B904DEC91 /* de.txt */; }; @@ -615,10 +641,12 @@ 996786038E31C671B650DFE4 /* SuggestionOverlayViews.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9720913FECB22A9A28C96333 /* SuggestionOverlayViews.swift */; }; 9973763A9F4EA4D8B4AE59EB /* SuggestionEngineRouter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 384FBCF5D7A3A446C5BE2B8D /* SuggestionEngineRouter.swift */; }; 998168DC04A6A13D7D1F3165 /* ModelDownloadManagerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 03CA4BBA3C54F840546033E0 /* ModelDownloadManagerTests.swift */; }; + 99FF57753B69D792A3B4B4B6 /* MLXToolchainProvisioner.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9CFBBF67EDCC6D7E19FFD04C /* MLXToolchainProvisioner.swift */; }; 9A2D50EF4911E45EEB4556D6 /* Aria2OutputParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = 83457CCF1A50CE83428C363D /* Aria2OutputParser.swift */; }; 9A55EDAF0F5D5127A39351C5 /* ContextBufferNavigationTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 024DDE8C1CE9BF00DE055990 /* ContextBufferNavigationTests.swift */; }; 9AAD623DEBAD8AF488C37818 /* CredentialFieldDetector.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8D9E5C3F2354CC40C0B2051B /* CredentialFieldDetector.swift */; }; 9AE3398FB0E4696C89550C04 /* EmojiMatcher.swift in Sources */ = {isa = PBXBuildFile; fileRef = EA8311FAC345FE431FA89855 /* EmojiMatcher.swift */; }; + 9B59E38D1A606272C121EB51 /* MLXHealthProbe.swift in Sources */ = {isa = PBXBuildFile; fileRef = E2E6128CA72E5FE4BA856229 /* MLXHealthProbe.swift */; }; 9B6C176547D2B6D118572E41 /* BaseCompletionPromptRenderer.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1C1AE4120FA68524700C9324 /* BaseCompletionPromptRenderer.swift */; }; 9B7FE4C9ED6959A6D5181EF5 /* EngineAndModelPaneView+Endpoint.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0A184538FD926947EBB88D9E /* EngineAndModelPaneView+Endpoint.swift */; }; 9BA9E52F33F00E65692EE6CD /* he-100k.txt in Resources */ = {isa = PBXBuildFile; fileRef = 7C9BB65FA5FC42B89766B037 /* he-100k.txt */; }; @@ -638,6 +666,7 @@ 9F8656C10B45DA6699A43787 /* TypefaceMatcher.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6E5BA02F079AD5EB20CDB215 /* TypefaceMatcher.swift */; }; A061CB3EEB9ABEF16CC93422 /* LICENSE in Resources */ = {isa = PBXBuildFile; fileRef = AC03B99F8F0C435522D44238 /* LICENSE */; }; A06D86F7475984F7AD810EFC /* InputModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = A29DF1C68154C21E6943D38E /* InputModels.swift */; }; + A06F4F2F50F4965811921312 /* MLXModelDownloaderTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E61D4E8C13EAD5806226C2C8 /* MLXModelDownloaderTests.swift */; }; A145F5E9E5BEC53EB46A596E /* PhrasePredictionScreenContextTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 74D4296C2F9A14E96AD86D7F /* PhrasePredictionScreenContextTests.swift */; }; A16DEE62FB45155A10E5DF74 /* SuggestionCoordinator+HostMarkedText.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8DAD5637347E83DDCF515568 /* SuggestionCoordinator+HostMarkedText.swift */; }; A1712224D1E7273EDA047544 /* SuggestionCoordinatorWordCompletionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 867188F95EB41AB11AA8862B /* SuggestionCoordinatorWordCompletionTests.swift */; }; @@ -649,6 +678,8 @@ A275EDDC0B43865C587C33AA /* TrailingDuplicationFilter.swift in Sources */ = {isa = PBXBuildFile; fileRef = AD851386E85D3ED0A1F17E47 /* TrailingDuplicationFilter.swift */; }; A29869C9E187C7EEF756E08F /* CaretTokenPositionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 24A7F2B20877E19BD2B27D1E /* CaretTokenPositionTests.swift */; }; A2993441AFFF43C25193BC98 /* SuggestionEngineSelectionPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 83256754ED5DD82BD749D6CE /* SuggestionEngineSelectionPolicy.swift */; }; + A2BB842F89BAEEEC91DED837 /* MLXGeneratorFailure.swift in Sources */ = {isa = PBXBuildFile; fileRef = 311D0D0F37A113911CAC1D07 /* MLXGeneratorFailure.swift */; }; + A2C7AE3FC56944880BE6B50A /* MLXServerLogWriter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 578A470C822BB93EEC442960 /* MLXServerLogWriter.swift */; }; A2E77B6BE41060407B2C0508 /* SwiftLog-Apache-2.0.txt in Resources */ = {isa = PBXBuildFile; fileRef = 19B257134CCCD0D4370E5287 /* SwiftLog-Apache-2.0.txt */; }; A2FCC90388ABD94CB3C9F9AA /* SuggestionOverlayAcceptanceTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4850AE68E84879254CBFEDE6 /* SuggestionOverlayAcceptanceTests.swift */; }; A30F28CB21A2DDE9E44FC10A /* BundleVersion.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1A951C04DF783B1F1EFDD1E5 /* BundleVersion.swift */; }; @@ -689,6 +720,7 @@ AB1B25E213AE0A79C6993239 /* HuggingFaceModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = C736C539D4C97B5664E498BE /* HuggingFaceModels.swift */; }; AB5D37BA744546F14C5566E8 /* AppearancePaneView.swift in Sources */ = {isa = PBXBuildFile; fileRef = AFBE491B3CA04FE9069B7B0F /* AppearancePaneView.swift */; }; AB8F186A7CB13779DAF729A0 /* MacroEngineTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 66D6D153C80A5BDE4905B81E /* MacroEngineTests.swift */; }; + AC376494710383408CB92DEB /* MLXToolchainRules.swift in Sources */ = {isa = PBXBuildFile; fileRef = 65674023B2B87D90B0B7878E /* MLXToolchainRules.swift */; }; AC3A30CDDFE6ED0A0D87005F /* FoundationModelDriftEvalTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2228A6C24CDE5F9D8DE33BF3 /* FoundationModelDriftEvalTests.swift */; }; AC424E6160AB68603F7B9A63 /* SuggestionCoordinator+TypingPrediction.swift in Sources */ = {isa = PBXBuildFile; fileRef = 57D3DEF846A089D523BB8BC0 /* SuggestionCoordinator+TypingPrediction.swift */; }; AC716016F8E26C1711C39F6D /* FileLogHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0C90C9EBCB70327D215EAE07 /* FileLogHandler.swift */; }; @@ -706,6 +738,7 @@ AF5AF5B7D32EDC0AF1A31557 /* InsertedTextAdvance.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6C8C1A4EC935BB837432BAF2 /* InsertedTextAdvance.swift */; }; AFFF05025897116EA6E58EA7 /* Aria2OutputParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = 83457CCF1A50CE83428C363D /* Aria2OutputParser.swift */; }; B00FDD3DEE0B73FF5136C91C /* FocusTracker.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5C9FDF029F7828CAF3FE8850 /* FocusTracker.swift */; }; + B090611E3D979A6C75303851 /* MLXCacheScanner.swift in Sources */ = {isa = PBXBuildFile; fileRef = E83059EDBC917FDE4F7AFE44 /* MLXCacheScanner.swift */; }; B0D7A0BC5621C8D23A9145B7 /* CustomRulesEditor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9E7D1184F0BC707F0A9148DD /* CustomRulesEditor.swift */; }; B16828E40F4694731120E6DC /* ModelDownloadTransferControls.swift in Sources */ = {isa = PBXBuildFile; fileRef = A25B3A7C488A6D863FE56113 /* ModelDownloadTransferControls.swift */; }; B1E93ED4D7C0B9BB77F95ADC /* CompositionInputModeClassifier.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9316289DD11A72D3EAC0A929 /* CompositionInputModeClassifier.swift */; }; @@ -714,6 +747,7 @@ B327BA8266E575FFF30AE288 /* DoubleTapAcceptanceState.swift in Sources */ = {isa = PBXBuildFile; fileRef = A7380A4057E7BC21F0977A07 /* DoubleTapAcceptanceState.swift */; }; B3706BDA93A1D8790CE8CC4A /* EmojiSynonymCatalogTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = D4DB988251D2B907F316A3B8 /* EmojiSynonymCatalogTests.swift */; }; B393275E40EA9EB901E20820 /* AppSurfaceClassifier.swift in Sources */ = {isa = PBXBuildFile; fileRef = 17A45844F33B55A3A58E9C71 /* AppSurfaceClassifier.swift */; }; + B39FB21A6FA1A214386EB433 /* MLXModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 70F9C7C2FD21D85CBFFE4807 /* MLXModels.swift */; }; B3E9C077CF064F2DEC3E22B8 /* ModelAndPresentationValueTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4BA1D28712B262ABD96F8399 /* ModelAndPresentationValueTests.swift */; }; B3EB0A706B7510CBB4AB23BB /* SettingsQuickLinkCard.swift in Sources */ = {isa = PBXBuildFile; fileRef = F1E69CFA319C5F9E4C30F699 /* SettingsQuickLinkCard.swift */; }; B413327721F83D9B5C8D4BCE /* SuggestionInteractionState.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6017E8AEDF88042933C61DFD /* SuggestionInteractionState.swift */; }; @@ -725,6 +759,7 @@ B5241C6026A756A7FDE4F041 /* LocalRuntimeResidencyPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4BB5841A73249EA3C1A4FA7E /* LocalRuntimeResidencyPolicy.swift */; }; B55B160E0534AE23BAC1C3DA /* CotabbyApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = CC1EDFB535AAA2EE0D67828A /* CotabbyApp.swift */; }; B582B229486BFA041A4641BA /* DateMacroEvaluatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E34A14D9D852BB9C6829DDC7 /* DateMacroEvaluatorTests.swift */; }; + B589039CB4C39E49C30E8816 /* MLXCacheScannerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9150E67CDE6F71584C8DFAA4 /* MLXCacheScannerTests.swift */; }; B5C255E58C128C4042FA7F66 /* PhrasePredictionScoringTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8E542E57459488F3D39A9053 /* PhrasePredictionScoringTests.swift */; }; B65B49F24F59154A7611FD22 /* HomePaneView.swift in Sources */ = {isa = PBXBuildFile; fileRef = D1123AB515110BD0CBA39490 /* HomePaneView.swift */; }; B695D8396E7FE07438DF9C9F /* TypingHistoryIndexTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 503A3986FC8C378991727721 /* TypingHistoryIndexTests.swift */; }; @@ -732,7 +767,9 @@ B6C2570DB8D17C8626D50F76 /* SuggestionCoordinatorContinuityTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5830E18C10728B7AF1DB2CCF /* SuggestionCoordinatorContinuityTests.swift */; }; B6F12EFB0AA3CC6BA8773476 /* CaretLinePosition.swift in Sources */ = {isa = PBXBuildFile; fileRef = BE5E9B2A196282738F8D4BAD /* CaretLinePosition.swift */; }; B71D51572125A69F279B3D79 /* llama-recall-cases.json in Resources */ = {isa = PBXBuildFile; fileRef = 78721FCF04E4D6F0C7C5DDCF /* llama-recall-cases.json */; }; + B72EC0CDA91CD50EA401688C /* MLXLiveEvalTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = F0DD6FD27894A80647739869 /* MLXLiveEvalTests.swift */; }; B76A1739BB3A656D6417C5D3 /* InkBaselineAnalyzerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BC7FA0146AC6B78725105BA4 /* InkBaselineAnalyzerTests.swift */; }; + B7EF26AC03B17D245B27F616 /* MLXServerDesireResolver.swift in Sources */ = {isa = PBXBuildFile; fileRef = DF574B2D5A31E00ECE381724 /* MLXServerDesireResolver.swift */; }; B803D0491F5CF19735F23B65 /* CurrencyEvaluator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 67332497BB5448AEDA700BA7 /* CurrencyEvaluator.swift */; }; B8B71602DFA0E6B7E61E2692 /* SuggestionSessionTypingTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = F0A61DC16ED76D694A59BF14 /* SuggestionSessionTypingTests.swift */; }; B9348E5D08C50B82D2C62BCF /* ContextBuffer.swift in Sources */ = {isa = PBXBuildFile; fileRef = C7B8727111208CB506C25C4F /* ContextBuffer.swift */; }; @@ -752,21 +789,25 @@ BD0AE666A1B4FBD2008F6B2A /* EmojiVariantResolver.swift in Sources */ = {isa = PBXBuildFile; fileRef = 721F5F358880C61A58AA3683 /* EmojiVariantResolver.swift */; }; BD4A3B059DBB46C791FF3FF7 /* EmojiUsageModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 68C4773B9316036F7E594EE1 /* EmojiUsageModels.swift */; }; BD4D2296BC699984D1916450 /* LanguageCatalog.swift in Sources */ = {isa = PBXBuildFile; fileRef = BE6EA89178F52F6D5B90CBFE /* LanguageCatalog.swift */; }; + BD9C9BDD38463AFAFE89EE6C /* MLXModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 70F9C7C2FD21D85CBFFE4807 /* MLXModels.swift */; }; BDC565D0F9830B3FFDA4252C /* EmojiUsageStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = 846A1C3328B3B1CB42E2E46B /* EmojiUsageStore.swift */; }; BDF13BDA6D30BB8FF755AC32 /* TypingPredictionCandidate.swift in Sources */ = {isa = PBXBuildFile; fileRef = C59EC159778E4802683A190E /* TypingPredictionCandidate.swift */; }; BE16C2851265AB659A322835 /* LlamaSuggestionEvalTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 31A62CE4DA3E55B5E117DBD8 /* LlamaSuggestionEvalTests.swift */; }; BEAFF8C487D70EA3EA4A1E0D /* CompletionSeamGuard.swift in Sources */ = {isa = PBXBuildFile; fileRef = F1A4C5DEA32868F99E953D99 /* CompletionSeamGuard.swift */; }; + BF0BD554ACCEE680CACEC108 /* MLXModelDownloader.swift in Sources */ = {isa = PBXBuildFile; fileRef = EF6D7EF81BDF13096889BED1 /* MLXModelDownloader.swift */; }; BFD3F43E9C83F3D0F7BF9ADE /* CalendarAccessibilityCapturePolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3FE7D19D22434E3E24804999 /* CalendarAccessibilityCapturePolicy.swift */; }; BFE1811C699B5B87530C3248 /* ControlTokenMarkers.swift in Sources */ = {isa = PBXBuildFile; fileRef = 33D4F378E5C70CAF7827A90A /* ControlTokenMarkers.swift */; }; C0468063222C5FEA7C1CFAB3 /* TypoCaseTransfer.swift in Sources */ = {isa = PBXBuildFile; fileRef = B293615FC411551C3FC15528 /* TypoCaseTransfer.swift */; }; C059BDAD7E8F82B058201A87 /* OnboardingTemplateRecommenderTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4FFD3AFACAAC963DAF59D153 /* OnboardingTemplateRecommenderTests.swift */; }; C080D50FA0ED1F32C92BCC36 /* HuggingFaceModelsTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7D2E806C1915514651343055 /* HuggingFaceModelsTests.swift */; }; C0B833234748E82D3382631A /* emoji.json in Resources */ = {isa = PBXBuildFile; fileRef = C379D77029D6E88C8C1B9AF7 /* emoji.json */; }; + C0E2C78678E89DEEBA7C50F1 /* ResourceUsageSummary.swift in Sources */ = {isa = PBXBuildFile; fileRef = 110BCAD18AFB2D446E5A607D /* ResourceUsageSummary.swift */; }; C10A7F3968C1A0B24F1C479D /* SwiftLog-NOTICE.txt in Resources */ = {isa = PBXBuildFile; fileRef = 1788B4543337A9E8388B8F6F /* SwiftLog-NOTICE.txt */; }; C14049A1A7C33F8CADDB3675 /* EmojiUsageStoreTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = F3D991D3536ED989C89AF44D /* EmojiUsageStoreTests.swift */; }; C1ADF50B2D3A3A05CD0FFA68 /* LlamaEvalScoring.swift in Sources */ = {isa = PBXBuildFile; fileRef = 202086735E28B96BBCE55E6B /* LlamaEvalScoring.swift */; }; C1D40B6EE4EFB064F3F64958 /* CaretLinePosition.swift in Sources */ = {isa = PBXBuildFile; fileRef = BE5E9B2A196282738F8D4BAD /* CaretLinePosition.swift */; }; C2085C2B2C6F0926A1266867 /* TokenHealingBufferTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 80588CA72AB588807BE1C499 /* TokenHealingBufferTests.swift */; }; + C20A1703F0DB96E48921C624 /* MLXServerManager.swift in Sources */ = {isa = PBXBuildFile; fileRef = 33291D41B4D72725738C3562 /* MLXServerManager.swift */; }; C25DF7FAB2DE87680FDFD6EE /* CaretLinePositionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = FCE4F5CA49ED9988AC4F4C6C /* CaretLinePositionTests.swift */; }; C27B7576CDA64AEE7558F700 /* ShortcutResolver.swift in Sources */ = {isa = PBXBuildFile; fileRef = E83C6E965A3978D0EBFB1EA2 /* ShortcutResolver.swift */; }; C2CD418FED8EB332C6417AF9 /* TokenCountEstimator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8A502BFB13373D1931B082CD /* TokenCountEstimator.swift */; }; @@ -826,6 +867,7 @@ CED430DD80E4769920E3C0E5 /* FocusModelsTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6F7E1691BC93836AE2FA8931 /* FocusModelsTests.swift */; }; CEDE9C238AD33258B36DB509 /* WelcomeHeroDemo.swift in Sources */ = {isa = PBXBuildFile; fileRef = F155556C751FB074B31CCC64 /* WelcomeHeroDemo.swift */; }; CF04C28598AF9A437990C8A4 /* SurfaceContextComposerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 652973AF18C20D51F81F76C5 /* SurfaceContextComposerTests.swift */; }; + CF0FCA57D1CBC6379147960E /* MenuResourceMonitorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 493009C5372CBBA5A6C9E171 /* MenuResourceMonitorTests.swift */; }; CF2EADEEEF5AA63FB9B9EA8E /* SuggestionInserter.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0A3D1125B962CBE0269EEDDB /* SuggestionInserter.swift */; }; CF391BC4A3C80C41048D64D1 /* TypoCaseTransfer.swift in Sources */ = {isa = PBXBuildFile; fileRef = B293615FC411551C3FC15528 /* TypoCaseTransfer.swift */; }; CFCBA5D14335960CF994F40D /* GhostFontSizeLimits.swift in Sources */ = {isa = PBXBuildFile; fileRef = EAC61CA8FEFB45FB1CCA453A /* GhostFontSizeLimits.swift */; }; @@ -873,6 +915,7 @@ D81AC2294A64E7A95B29E081 /* fr.txt in Resources */ = {isa = PBXBuildFile; fileRef = 5DCA3E46621815A19300365F /* fr.txt */; }; D8B0D41FED19138D1869BB3C /* EmojiRecents.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DE72A73D906285AE10F623B /* EmojiRecents.swift */; }; D8E0E5FA644F62103D18645B /* WordCountFormatterTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0854896090B568FFB1926D24 /* WordCountFormatterTests.swift */; }; + D8FC1912B9806CC1797397A0 /* MLXModelCompatibility.swift in Sources */ = {isa = PBXBuildFile; fileRef = C659AC4B22F9E2E0D69D137E /* MLXModelCompatibility.swift */; }; D98538D3C1344422354B0B2E /* SuggestionDebugLoggerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9A02F9151AEBBD825D64CEFA /* SuggestionDebugLoggerTests.swift */; }; D9BF921EDFAE90D4D15DA82F /* GhostTextColorPresetPaletteTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 47F6000E673651AE72243F8A /* GhostTextColorPresetPaletteTests.swift */; }; D9CAF205C283ACD79B70333F /* GPUStatisticsReaderTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = D08DF3F7E58429007C599247 /* GPUStatisticsReaderTests.swift */; }; @@ -898,6 +941,7 @@ DED8731A5A34A10349AA05A8 /* EmojiPickerModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8736D4321E8BF847D1F7338E /* EmojiPickerModels.swift */; }; DF38049917DC33C6235DB3E6 /* CaretTokenPosition.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9EBE00782EE6813AF3224CD5 /* CaretTokenPosition.swift */; }; E04950B45EF6C6CE47EE5081 /* MacroController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8A38CFCDC00DC4DB86D91BB2 /* MacroController.swift */; }; + E105B69FB9E82E68F636760B /* EngineAndModelPaneView+MLX.swift in Sources */ = {isa = PBXBuildFile; fileRef = 057A4153C112521EDCFD1CC2 /* EngineAndModelPaneView+MLX.swift */; }; E1A8EA5F00C6B2E228FC9ACB /* EmojiPopularityTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6C28F67F089B66EC18C24182 /* EmojiPopularityTests.swift */; }; E2A0998154C6BFFC69B54038 /* SuggestionStreamingStateTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6A0C2FF2D19C8280B8A65DC3 /* SuggestionStreamingStateTests.swift */; }; E2E830C1D5FEFE5F621018FE /* EmojiUsageModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 68C4773B9316036F7E594EE1 /* EmojiUsageModels.swift */; }; @@ -915,6 +959,7 @@ E53332E91F6520C7FAEEA591 /* EmojiQueryRun.swift in Sources */ = {isa = PBXBuildFile; fileRef = 264C3F355F2F82D1964DD263 /* EmojiQueryRun.swift */; }; E5B8A88A1BC2F84444EABDE0 /* VisualContextCoordinatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 40D912ACDEDD1EE3589180CB /* VisualContextCoordinatorTests.swift */; }; E5CF3CFC00B6607AE8D62FED /* EmojiRecents.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DE72A73D906285AE10F623B /* EmojiRecents.swift */; }; + E606A8F90DD5A91CE72581AC /* MLXToolchainProvisionerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8617EB6F68EF450F58DFD59B /* MLXToolchainProvisionerTests.swift */; }; E60DE8299D729AB113647904 /* FocusSnapshotResolverLiveTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 110F737140F015E1A18E5A58 /* FocusSnapshotResolverLiveTests.swift */; }; E6650273045E6F1976AF23CA /* EngineAndModelPaneView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9C35A0A68DA5D2B59460A67E /* EngineAndModelPaneView.swift */; }; E68459DC91D2F70D9D3D1A40 /* TypingHistoryScrubber.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3880F8FF215912CD4C337F8E /* TypingHistoryScrubber.swift */; }; @@ -922,6 +967,7 @@ E6A96157E8689F4762D83AC8 /* Llama.cpp-MIT.txt in Resources */ = {isa = PBXBuildFile; fileRef = 617B396FAD84E6F4A270B5EA /* Llama.cpp-MIT.txt */; }; E6C99AB3E1A56E71F3A518FB /* ModelDownloadManager.swift in Sources */ = {isa = PBXBuildFile; fileRef = 050DC9D858A124FD1CFA4D99 /* ModelDownloadManager.swift */; }; E6DD9EAAFF4001E2922B5F55 /* OnboardingStyle.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9ABC808DE01AF9AD04E38943 /* OnboardingStyle.swift */; }; + E7206CC66B0FDE5933BE2CF9 /* MLXToolchainRules.swift in Sources */ = {isa = PBXBuildFile; fileRef = 65674023B2B87D90B0B7878E /* MLXToolchainRules.swift */; }; E72C52902EC84711A80C6D25 /* KeyRecorderView.swift in Sources */ = {isa = PBXBuildFile; fileRef = A1D44E50290A78AFAE7753DB /* KeyRecorderView.swift */; }; E79834E768C2001098CE6132 /* LocalRuntimeResidencyPolicyTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = C0CF171FCE0BD5D84E555F1E /* LocalRuntimeResidencyPolicyTests.swift */; }; E83E989C5132F2A8C8FED8EB /* SuggestionSettingsDomainTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2E7C883990AE384602BA9AF6 /* SuggestionSettingsDomainTests.swift */; }; @@ -963,6 +1009,7 @@ F0A97143C565D2C2BD2339DF /* OverlayControllerTypefaceTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = EB446CE06049A4F958C9DC91 /* OverlayControllerTypefaceTests.swift */; }; F0DCEFED640B453A5ECEB810 /* HuggingFaceModelBrowserView.swift in Sources */ = {isa = PBXBuildFile; fileRef = C696204AAEBA357129F77646 /* HuggingFaceModelBrowserView.swift */; }; F0E47B759322F911BB54EFCB /* EmojiMatcher.swift in Sources */ = {isa = PBXBuildFile; fileRef = EA8311FAC345FE431FA89855 /* EmojiMatcher.swift */; }; + F0F06AF3A0508BE380AD3CAD /* SuggestionSettingsMLXTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 93208561AC9A2CEB4F18D6C0 /* SuggestionSettingsMLXTests.swift */; }; F10E203703E57C0EF2C0F98E /* StaticTextRunWalkThrottle.swift in Sources */ = {isa = PBXBuildFile; fileRef = 21E26C899DE4FE7AA9CC7869 /* StaticTextRunWalkThrottle.swift */; }; F13B85DD24398CD22C3560C1 /* InputModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = A29DF1C68154C21E6943D38E /* InputModels.swift */; }; F140C1894C54E0A94EAE70BC /* BundledRuntimeLocator.swift in Sources */ = {isa = PBXBuildFile; fileRef = D1AB5EBB44EE64EB3EB1426F /* BundledRuntimeLocator.swift */; }; @@ -977,6 +1024,8 @@ F38776739B0987C0E3F2D343 /* SettingsSearchRankerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 95C62A4403E4815004C6FCB8 /* SettingsSearchRankerTests.swift */; }; F3B72E0E3C700E57BC3B48F8 /* EmojiTriggerStateMachine.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8159F4DD2F03935DD17BD907 /* EmojiTriggerStateMachine.swift */; }; F3DA1B242F30332AA91C00C5 /* FoundationModelSuggestionEngineTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 377A0BBB59988043005A138A /* FoundationModelSuggestionEngineTests.swift */; }; + F3E29226CEF2792782E558B3 /* MLXServerLogFilterTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7ECD66E36EB8314C5A11A795 /* MLXServerLogFilterTests.swift */; }; + F3F9E720ECF623E5A7F631CA /* MLXToolchainInstaller.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9093E08339428C7A0107D993 /* MLXToolchainInstaller.swift */; }; F3FC8AEB8504F7D1BB1C6443 /* DisplayCoordinateConverter.swift in Sources */ = {isa = PBXBuildFile; fileRef = ECC1416A95346CB72BE036FE /* DisplayCoordinateConverter.swift */; }; F40ADF32D2C23D6031069B67 /* SuggestionRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2E23EF696D34CFB8EF320668 /* SuggestionRequest.swift */; }; F45DC37FC763FB932B9AB3FB /* EngineAndModelPaneView+OpenSource.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8B2C3A2CEC6E0A13E2102D15 /* EngineAndModelPaneView+OpenSource.swift */; }; @@ -1004,6 +1053,7 @@ FC1757BF6BF6636C5581A373 /* FocusedInputContext.swift in Sources */ = {isa = PBXBuildFile; fileRef = CFB06531A54FDCDBAAD14E8B /* FocusedInputContext.swift */; }; FC4706CDB470537FE069FF22 /* DisplayCoordinateConverter.swift in Sources */ = {isa = PBXBuildFile; fileRef = ECC1416A95346CB72BE036FE /* DisplayCoordinateConverter.swift */; }; FC61422DC8E8EBD10A8F439B /* TypingHistoryModels.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7CFDC85F1E37BF37FB4751A1 /* TypingHistoryModels.swift */; }; + FC9590FCC5F594D52836F78B /* MLXModelDownloader.swift in Sources */ = {isa = PBXBuildFile; fileRef = EF6D7EF81BDF13096889BED1 /* MLXModelDownloader.swift */; }; FCB060F562093AACA90CAA21 /* MarkerSelectionSynthesizer.swift in Sources */ = {isa = PBXBuildFile; fileRef = A9FD2044CDEA95EC02C9B2FE /* MarkerSelectionSynthesizer.swift */; }; FCC571EC239846F06007BFCA /* CotabbyAppEnvironment.swift in Sources */ = {isa = PBXBuildFile; fileRef = 711293EA57808B9428C7B908 /* CotabbyAppEnvironment.swift */; }; FCD81796FE4DC55778D57686 /* ConfidenceSuppressionPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 122298AE151ECEEC175878BF /* ConfidenceSuppressionPolicy.swift */; }; @@ -1045,11 +1095,13 @@ 0397F1DACB094A0F6A66BC0E /* it.txt */ = {isa = PBXFileReference; lastKnownFileType = text; path = it.txt; sourceTree = ""; }; 03CA4BBA3C54F840546033E0 /* ModelDownloadManagerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ModelDownloadManagerTests.swift; sourceTree = ""; }; 04364C3242A25358E7C85FD6 /* ClipboardRelevanceFilterTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ClipboardRelevanceFilterTests.swift; sourceTree = ""; }; + 044F1B6638860E6E8C37B5C6 /* MLXServerLogFilter.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXServerLogFilter.swift; sourceTree = ""; }; 049102F0ED85618106A042C4 /* CalendarAccessibilityCaptureGuard.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CalendarAccessibilityCaptureGuard.swift; sourceTree = ""; }; 04F95814016D80E0A23974F0 /* ActiveSuggestionSession.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ActiveSuggestionSession.swift; sourceTree = ""; }; 050837EC77D1F4290AC13479 /* BundleVersionTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = BundleVersionTests.swift; sourceTree = ""; }; 050DC9D858A124FD1CFA4D99 /* ModelDownloadManager.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ModelDownloadManager.swift; sourceTree = ""; }; 054987E76CA9D1FA4F81EA8F /* SOURCES.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = SOURCES.md; sourceTree = ""; }; + 057A4153C112521EDCFD1CC2 /* EngineAndModelPaneView+MLX.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "EngineAndModelPaneView+MLX.swift"; sourceTree = ""; }; 05F173F93FFA5FFBC169DA25 /* RequestID.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RequestID.swift; sourceTree = ""; }; 06FC72BD286ABA5B3CEF82F7 /* SentenceBoundaryClassifier.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SentenceBoundaryClassifier.swift; sourceTree = ""; }; 07480CE96ED0EBD94817C6B1 /* GeneralPaneView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GeneralPaneView.swift; sourceTree = ""; }; @@ -1065,6 +1117,7 @@ 0DB8BFC8B80A47088FDDFC24 /* TypingHistoryPhraseEngine.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TypingHistoryPhraseEngine.swift; sourceTree = ""; }; 0F328B43743D803A16045110 /* HostFontRegistry.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HostFontRegistry.swift; sourceTree = ""; }; 10107090891BE11E2B34BC9E /* TypefaceMatcherTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TypefaceMatcherTests.swift; sourceTree = ""; }; + 110BCAD18AFB2D446E5A607D /* ResourceUsageSummary.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ResourceUsageSummary.swift; sourceTree = ""; }; 110F737140F015E1A18E5A58 /* FocusSnapshotResolverLiveTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FocusSnapshotResolverLiveTests.swift; sourceTree = ""; }; 11CF768650A90705FC0D2730 /* SuggestionTextColorCodec.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionTextColorCodec.swift; sourceTree = ""; }; 12082948AEBC0DFE5ADC6961 /* SuggestionAvailabilityEvaluator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionAvailabilityEvaluator.swift; sourceTree = ""; }; @@ -1095,6 +1148,7 @@ 1B966EA885CB107161FC3006 /* SystemMetricsStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SystemMetricsStore.swift; sourceTree = ""; }; 1BB42779285A1BA40B7848C8 /* TickMarkSlider.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TickMarkSlider.swift; sourceTree = ""; }; 1BB44E9DE070D5EFC52B968A /* CaretGeometrySelectorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CaretGeometrySelectorTests.swift; sourceTree = ""; }; + 1BC7B3FAE8ABAFE9910AEED2 /* MenuResourceMonitor.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MenuResourceMonitor.swift; sourceTree = ""; }; 1BF6C6A11D5B5A7B987AFB9C /* TextLayoutCaretEstimator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TextLayoutCaretEstimator.swift; sourceTree = ""; }; 1C1AE4120FA68524700C9324 /* BaseCompletionPromptRenderer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = BaseCompletionPromptRenderer.swift; sourceTree = ""; }; 1D1AC5EC06664F49A6AE2B17 /* SettingsNavigationModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsNavigationModel.swift; sourceTree = ""; }; @@ -1119,6 +1173,7 @@ 21E26C899DE4FE7AA9CC7869 /* StaticTextRunWalkThrottle.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = StaticTextRunWalkThrottle.swift; sourceTree = ""; }; 2228A6C24CDE5F9D8DE33BF3 /* FoundationModelDriftEvalTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FoundationModelDriftEvalTests.swift; sourceTree = ""; }; 225D56594DCCBEAE2BD25E8D /* phrase-prediction-1337.json */ = {isa = PBXFileReference; lastKnownFileType = text.json; path = "phrase-prediction-1337.json"; sourceTree = ""; }; + 2289DB842350106360F00DB9 /* MLXServerManagerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXServerManagerTests.swift; sourceTree = ""; }; 22FA75F9AD318B9084B23ECF /* PostExhaustionAcceptanceState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PostExhaustionAcceptanceState.swift; sourceTree = ""; }; 2471F967A05EB4C8BF7CD242 /* SuggestionCoordinatorRigTestCase.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionCoordinatorRigTestCase.swift; sourceTree = ""; }; 2472C76E96009300EBF038D4 /* CotabbyInference-MIT.txt */ = {isa = PBXFileReference; lastKnownFileType = text; path = "CotabbyInference-MIT.txt"; sourceTree = ""; }; @@ -1131,6 +1186,7 @@ 262BE2F1E97389FE8D7A5FB9 /* Cotabby.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = Cotabby.app; sourceTree = BUILT_PRODUCTS_DIR; }; 264C3F355F2F82D1964DD263 /* EmojiQueryRun.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiQueryRun.swift; sourceTree = ""; }; 266C8BF167118855C56FC7F4 /* BrowserAppDetector.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = BrowserAppDetector.swift; sourceTree = ""; }; + 27049BD6A6B9E46DE07E01C8 /* MLXServerDesireResolverTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXServerDesireResolverTests.swift; sourceTree = ""; }; 27B03589D82D0ADCF41B9CD0 /* SurfaceContextComposer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SurfaceContextComposer.swift; sourceTree = ""; }; 29163A3D0BB0F11EEAC2985A /* GhostTextPanelView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GhostTextPanelView.swift; sourceTree = ""; }; 2B23950F38B1B3C8A5CC5BE5 /* OpenAICompatibleCredentialStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OpenAICompatibleCredentialStore.swift; sourceTree = ""; }; @@ -1156,8 +1212,10 @@ 30452E10F080145EAD9F3667 /* LlamaSuggestionEngineTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LlamaSuggestionEngineTests.swift; sourceTree = ""; }; 308F8660F70ED64511BDB59F /* RandomMacroEvaluator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RandomMacroEvaluator.swift; sourceTree = ""; }; 30AB3A99BEB2EA978298FC6F /* MidWordContinuationPolicyTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MidWordContinuationPolicyTests.swift; sourceTree = ""; }; + 311D0D0F37A113911CAC1D07 /* MLXGeneratorFailure.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXGeneratorFailure.swift; sourceTree = ""; }; 3153FB925190B70E0502C43D /* InkCaretAnalyzerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InkCaretAnalyzerTests.swift; sourceTree = ""; }; 31A62CE4DA3E55B5E117DBD8 /* LlamaSuggestionEvalTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LlamaSuggestionEvalTests.swift; sourceTree = ""; }; + 33291D41B4D72725738C3562 /* MLXServerManager.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXServerManager.swift; sourceTree = ""; }; 3350EDE01ED5125520C79D53 /* SettingsCoordinator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsCoordinator.swift; sourceTree = ""; }; 338EE4D92F8BE01CF43FE876 /* OllamaPreloadWorkControllerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OllamaPreloadWorkControllerTests.swift; sourceTree = ""; }; 33D4F378E5C70CAF7827A90A /* ControlTokenMarkers.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ControlTokenMarkers.swift; sourceTree = ""; }; @@ -1169,6 +1227,7 @@ 365AE69E4E1A3DD3486D203A /* CredentialFieldDetectorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CredentialFieldDetectorTests.swift; sourceTree = ""; }; 36652DB88C5948AA4A31524A /* ModelFileValidator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ModelFileValidator.swift; sourceTree = ""; }; 3680E1B8FA712A888F509640 /* ClipboardContentDistillerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ClipboardContentDistillerTests.swift; sourceTree = ""; }; + 36C581E9E992813892959DBE /* MLXEngineWiringTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXEngineWiringTests.swift; sourceTree = ""; }; 377A0BBB59988043005A138A /* FoundationModelSuggestionEngineTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FoundationModelSuggestionEngineTests.swift; sourceTree = ""; }; 384D85534F3C2F40E2FD8781 /* CompletionContentPolicy.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CompletionContentPolicy.swift; sourceTree = ""; }; 384FBCF5D7A3A446C5BE2B8D /* SuggestionEngineRouter.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionEngineRouter.swift; sourceTree = ""; }; @@ -1208,6 +1267,7 @@ 4850AE68E84879254CBFEDE6 /* SuggestionOverlayAcceptanceTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionOverlayAcceptanceTests.swift; sourceTree = ""; }; 4861CC2533F3EA222A0833A9 /* SuggestionAnchorCache.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionAnchorCache.swift; sourceTree = ""; }; 487D8C5521411FC6845D4A51 /* FocusSessionScopedCacheTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FocusSessionScopedCacheTests.swift; sourceTree = ""; }; + 493009C5372CBBA5A6C9E171 /* MenuResourceMonitorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MenuResourceMonitorTests.swift; sourceTree = ""; }; 49DB247AACE1B28D45DB52B0 /* SuggestionOverlayStabilityGate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionOverlayStabilityGate.swift; sourceTree = ""; }; 4A51B1559F4CE6728D39C959 /* VisualContextModels.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VisualContextModels.swift; sourceTree = ""; }; 4A65BCABF4B23690AE3DBE93 /* PixelCaretLocator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PixelCaretLocator.swift; sourceTree = ""; }; @@ -1241,6 +1301,7 @@ 56A2F75C66BECA61A722D3FB /* LLMIOFileHandler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LLMIOFileHandler.swift; sourceTree = ""; }; 56B8D2232F271197468CBC11 /* FocusCapabilityResolver.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FocusCapabilityResolver.swift; sourceTree = ""; }; 571B221ACAFED86DABB10F48 /* SelfCaptureGateTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SelfCaptureGateTests.swift; sourceTree = ""; }; + 578A470C822BB93EEC442960 /* MLXServerLogWriter.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXServerLogWriter.swift; sourceTree = ""; }; 57B29DAC600626783649475A /* SettingsSearchRanker.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsSearchRanker.swift; sourceTree = ""; }; 57D3DEF846A089D523BB8BC0 /* SuggestionCoordinator+TypingPrediction.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "SuggestionCoordinator+TypingPrediction.swift"; sourceTree = ""; }; 57F9A25D8935B8C2156D7CE5 /* SyntheticReplacePlannerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SyntheticReplacePlannerTests.swift; sourceTree = ""; }; @@ -1278,6 +1339,7 @@ 64B0584FA16465453F04CA8C /* OCRTextHygieneTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OCRTextHygieneTests.swift; sourceTree = ""; }; 652973AF18C20D51F81F76C5 /* SurfaceContextComposerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SurfaceContextComposerTests.swift; sourceTree = ""; }; 655F56C230F45CAC13EEE59C /* SuggestionTextColorCodecTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionTextColorCodecTests.swift; sourceTree = ""; }; + 65674023B2B87D90B0B7878E /* MLXToolchainRules.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXToolchainRules.swift; sourceTree = ""; }; 6586D6D735F287F43D42520A /* DeviceInfo.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DeviceInfo.swift; sourceTree = ""; }; 65AF0FF1CE97A0761ADAFEB8 /* InlineCommandCoordinatorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InlineCommandCoordinatorTests.swift; sourceTree = ""; }; 65E6587C61CBD8D64E852EBF /* SuggestionCoordinatorDoubleTapTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionCoordinatorDoubleTapTests.swift; sourceTree = ""; }; @@ -1321,6 +1383,7 @@ 705E9119B1661BF464496D39 /* CompletionRenderModePolicy.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CompletionRenderModePolicy.swift; sourceTree = ""; }; 70C489EC7EA50C2B103A6B09 /* Sparkle-LICENSE.txt */ = {isa = PBXFileReference; lastKnownFileType = text; path = "Sparkle-LICENSE.txt"; sourceTree = ""; }; 70F68E9A9255F607C4439ED5 /* VisualContextStartCoalescerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VisualContextStartCoalescerTests.swift; sourceTree = ""; }; + 70F9C7C2FD21D85CBFFE4807 /* MLXModels.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXModels.swift; sourceTree = ""; }; 7103021AEC287186CE25415D /* HostFaceMemory.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HostFaceMemory.swift; sourceTree = ""; }; 711293EA57808B9428C7B908 /* CotabbyAppEnvironment.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CotabbyAppEnvironment.swift; sourceTree = ""; }; 7113D3373525113CA69E7597 /* PermissionsPaneView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PermissionsPaneView.swift; sourceTree = ""; }; @@ -1348,6 +1411,7 @@ 79AAAB1C46A0E5A9C6FC7D3D /* SettingsAttentionEvaluator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsAttentionEvaluator.swift; sourceTree = ""; }; 7AEBBBAB9588AA8EDBA41520 /* PermissionAndContextModelTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PermissionAndContextModelTests.swift; sourceTree = ""; }; 7B3AD6D4125ED99669F4EC32 /* DownloadOutcomeClassifierTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DownloadOutcomeClassifierTests.swift; sourceTree = ""; }; + 7B70ED445F6B46B9FE4176A5 /* SuggestionEngineKindTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionEngineKindTests.swift; sourceTree = ""; }; 7B9C30F20BE1D9C3B0AF8A9E /* SuggestionRequestFactoryTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionRequestFactoryTests.swift; sourceTree = ""; }; 7C9BB65FA5FC42B89766B037 /* he-100k.txt */ = {isa = PBXFileReference; lastKnownFileType = text; path = "he-100k.txt"; sourceTree = ""; }; 7CC0C7FE7FB134C225EA7B71 /* CompletionSeamGuardTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CompletionSeamGuardTests.swift; sourceTree = ""; }; @@ -1355,9 +1419,12 @@ 7D2E806C1915514651343055 /* HuggingFaceModelsTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HuggingFaceModelsTests.swift; sourceTree = ""; }; 7D2F4ED8C7493EF4B967BFF9 /* Aria2Provisioner.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Aria2Provisioner.swift; sourceTree = ""; }; 7DE038A6DF4A16B3EB5CEBD3 /* MenuBarView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MenuBarView.swift; sourceTree = ""; }; + 7E348E1CB10B4EAAC9CA1572 /* ResourceUsageSummaryTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ResourceUsageSummaryTests.swift; sourceTree = ""; }; + 7ECD66E36EB8314C5A11A795 /* MLXServerLogFilterTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXServerLogFilterTests.swift; sourceTree = ""; }; 7F19230243E3B5DE5A76244C /* SignOffCueTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SignOffCueTests.swift; sourceTree = ""; }; 7F7E66CFA5AB733CE1F793A8 /* MarkerSelectionSynthesizerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MarkerSelectionSynthesizerTests.swift; sourceTree = ""; }; 7FB5276B7DDE61030A57EF9A /* TypingHistoryPhrasePredictorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TypingHistoryPhrasePredictorTests.swift; sourceTree = ""; }; + 803C81D876465AAAD03691AB /* MLXModelsTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXModelsTests.swift; sourceTree = ""; }; 80588CA72AB588807BE1C499 /* TokenHealingBufferTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TokenHealingBufferTests.swift; sourceTree = ""; }; 8159F4DD2F03935DD17BD907 /* EmojiTriggerStateMachine.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiTriggerStateMachine.swift; sourceTree = ""; }; 816106CA251AFDE69A804762 /* CompletionRenderModeTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CompletionRenderModeTests.swift; sourceTree = ""; }; @@ -1371,6 +1438,7 @@ 846A1C3328B3B1CB42E2E46B /* EmojiUsageStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiUsageStore.swift; sourceTree = ""; }; 8543EBA09E11A5066880CC91 /* SuggestionResult.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionResult.swift; sourceTree = ""; }; 85BF316556FDA64CB8AD07B6 /* PermissionManager.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PermissionManager.swift; sourceTree = ""; }; + 8617EB6F68EF450F58DFD59B /* MLXToolchainProvisionerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXToolchainProvisionerTests.swift; sourceTree = ""; }; 863B5A1DC3C4B9668F620245 /* CalendarAccessibilityCapturePolicyTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CalendarAccessibilityCapturePolicyTests.swift; sourceTree = ""; }; 867188F95EB41AB11AA8862B /* SuggestionCoordinatorWordCompletionTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionCoordinatorWordCompletionTests.swift; sourceTree = ""; }; 86E1C98557E3E592AF8B6735 /* PerAppShortcutOverrideTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PerAppShortcutOverrideTests.swift; sourceTree = ""; }; @@ -1388,6 +1456,7 @@ 8AF81CAD55A14DC378DD5756 /* SettingsIndexTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsIndexTests.swift; sourceTree = ""; }; 8B2C3A2CEC6E0A13E2102D15 /* EngineAndModelPaneView+OpenSource.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "EngineAndModelPaneView+OpenSource.swift"; sourceTree = ""; }; 8B4F6E70B8A242F7BDE5361A /* FocusDebugOverlayController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FocusDebugOverlayController.swift; sourceTree = ""; }; + 8B87FBEAF425DEE74F008EE5 /* MLXToolchainRulesTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXToolchainRulesTests.swift; sourceTree = ""; }; 8BE5F414704A8264C2946A50 /* TypoGateTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TypoGateTests.swift; sourceTree = ""; }; 8C151BF4D39485E5CFACFECB /* ApplicationBundleMetadataTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ApplicationBundleMetadataTests.swift; sourceTree = ""; }; 8D881FED12A85FFC20F2C9D7 /* DisplayCoordinateConverterTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DisplayCoordinateConverterTests.swift; sourceTree = ""; }; @@ -1395,6 +1464,8 @@ 8DAD5637347E83DDCF515568 /* SuggestionCoordinator+HostMarkedText.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "SuggestionCoordinator+HostMarkedText.swift"; sourceTree = ""; }; 8E542E57459488F3D39A9053 /* PhrasePredictionScoringTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PhrasePredictionScoringTests.swift; sourceTree = ""; }; 8E89746E8CE7E9487337EE6F /* InsertionSafetyGate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InsertionSafetyGate.swift; sourceTree = ""; }; + 9093E08339428C7A0107D993 /* MLXToolchainInstaller.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXToolchainInstaller.swift; sourceTree = ""; }; + 9150E67CDE6F71584C8DFAA4 /* MLXCacheScannerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXCacheScannerTests.swift; sourceTree = ""; }; 9227DE16CA5CE2E085C1A891 /* PerAppShortcutOverrideStoreTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PerAppShortcutOverrideStoreTests.swift; sourceTree = ""; }; 924714B1688E884D9AB2A79D /* Aria2LocatorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Aria2LocatorTests.swift; sourceTree = ""; }; 924CAA5E25C596A9FAB7602B /* InlinePreviewPanelController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InlinePreviewPanelController.swift; sourceTree = ""; }; @@ -1402,6 +1473,7 @@ 92854D0927EBF17A48AE2804 /* LowPowerModeMonitorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LowPowerModeMonitorTests.swift; sourceTree = ""; }; 928A884292355FEF8B605B64 /* SuggestionFadeInPolicyTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionFadeInPolicyTests.swift; sourceTree = ""; }; 9316289DD11A72D3EAC0A929 /* CompositionInputModeClassifier.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CompositionInputModeClassifier.swift; sourceTree = ""; }; + 93208561AC9A2CEB4F18D6C0 /* SuggestionSettingsMLXTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionSettingsMLXTests.swift; sourceTree = ""; }; 94A26D86030FC5FA67371C52 /* DateMacroEvaluator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DateMacroEvaluator.swift; sourceTree = ""; }; 94B45E958CF9BE63A728E273 /* TypefaceEvidenceTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TypefaceEvidenceTests.swift; sourceTree = ""; }; 95C62A4403E4815004C6FCB8 /* SettingsSearchRankerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsSearchRankerTests.swift; sourceTree = ""; }; @@ -1421,6 +1493,7 @@ 9BB500911B047A5363D51E71 /* LLMIOFileHandlerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LLMIOFileHandlerTests.swift; sourceTree = ""; }; 9C35A0A68DA5D2B59460A67E /* EngineAndModelPaneView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EngineAndModelPaneView.swift; sourceTree = ""; }; 9CCBD29CF7B0BA5A9CFDEEA3 /* DeepGeometryWalkThrottle.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DeepGeometryWalkThrottle.swift; sourceTree = ""; }; + 9CFBBF67EDCC6D7E19FFD04C /* MLXToolchainProvisioner.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXToolchainProvisioner.swift; sourceTree = ""; }; 9D598CC3134879999D567455 /* SuggestionOverlayPresenter.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionOverlayPresenter.swift; sourceTree = ""; }; 9DE86A49339BB507D197629D /* MenuBarPresentationObserver.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MenuBarPresentationObserver.swift; sourceTree = ""; }; 9E76C76AEE756F43787EEECE /* SuggestionCoordinator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionCoordinator.swift; sourceTree = ""; }; @@ -1462,6 +1535,7 @@ AAD02052E51AFE56BECB92E4 /* EmojiPickerControllerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiPickerControllerTests.swift; sourceTree = ""; }; AAE4180FC6B2E085CEE93F23 /* HostTextMetricsProbe.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HostTextMetricsProbe.swift; sourceTree = ""; }; AAE89BEAF57DF3A287DF56F1 /* PromptSectionBudget.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PromptSectionBudget.swift; sourceTree = ""; }; + AB4A82DFAD71F7568E801FB8 /* MLXProcessRunnerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXProcessRunnerTests.swift; sourceTree = ""; }; AB84C175EA5B59E118F58C49 /* llama-eval-cases.json */ = {isa = PBXFileReference; lastKnownFileType = text.json; path = "llama-eval-cases.json"; sourceTree = ""; }; AC03B99F8F0C435522D44238 /* LICENSE */ = {isa = PBXFileReference; path = LICENSE; sourceTree = ""; }; ACD93D7DE258C8E5367A433D /* EmojiCatalog.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiCatalog.swift; sourceTree = ""; }; @@ -1538,6 +1612,7 @@ C59EC159778E4802683A190E /* TypingPredictionCandidate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TypingPredictionCandidate.swift; sourceTree = ""; }; C5C0792BF971B702DE0A095E /* GhostTextLayout.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GhostTextLayout.swift; sourceTree = ""; }; C648EBB10D7F8E0B904DEC91 /* de.txt */ = {isa = PBXFileReference; lastKnownFileType = text; path = de.txt; sourceTree = ""; }; + C659AC4B22F9E2E0D69D137E /* MLXModelCompatibility.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXModelCompatibility.swift; sourceTree = ""; }; C696204AAEBA357129F77646 /* HuggingFaceModelBrowserView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HuggingFaceModelBrowserView.swift; sourceTree = ""; }; C6A19E3F68A42D2FAC95E000 /* SuggestionTextNormalizer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionTextNormalizer.swift; sourceTree = ""; }; C736C539D4C97B5664E498BE /* HuggingFaceModels.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HuggingFaceModels.swift; sourceTree = ""; }; @@ -1553,6 +1628,7 @@ CC1EDFB535AAA2EE0D67828A /* CotabbyApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CotabbyApp.swift; sourceTree = ""; }; CC30D79131D562C5CD7FA639 /* SuggestionSettingsModelDoubleTapTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionSettingsModelDoubleTapTests.swift; sourceTree = ""; }; CC5CD24D2FF35FD36B8D78DE /* SuggestionQualityMetricsStoreTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionQualityMetricsStoreTests.swift; sourceTree = ""; }; + CC8B01E696534BACF09ED6C7 /* MLXModelCompatibilityTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXModelCompatibilityTests.swift; sourceTree = ""; }; CD1FC547BE2E96D2ABB55DF4 /* LlamaSuggestionEngineStreamingTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LlamaSuggestionEngineStreamingTests.swift; sourceTree = ""; }; CE8B99181BF74C1F1DC5F6A7 /* HostTextMetricsCache.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HostTextMetricsCache.swift; sourceTree = ""; }; CEDD364CC7C995916BD1E4B0 /* TokenHealingPlanTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TokenHealingPlanTests.swift; sourceTree = ""; }; @@ -1609,11 +1685,14 @@ DE29E54D78A9DD7CD87626DD /* LanguageSupportTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LanguageSupportTests.swift; sourceTree = ""; }; DE6E2A19BF6B87A0C6FA9D49 /* Aria2DownloadService.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Aria2DownloadService.swift; sourceTree = ""; }; DEBD6113A3C1038BECC99245 /* PerformancePaneView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PerformancePaneView.swift; sourceTree = ""; }; + DECAC67D186C067A6F75517D /* MLXProcessRunner.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXProcessRunner.swift; sourceTree = ""; }; DEF8F1A40EAA82773C07F8B9 /* VisualContextStartCoalescer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VisualContextStartCoalescer.swift; sourceTree = ""; }; + DF574B2D5A31E00ECE381724 /* MLXServerDesireResolver.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXServerDesireResolver.swift; sourceTree = ""; }; DF5872EC7795CC1EFF6D0D04 /* EngineAndModelPaneView+Power.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "EngineAndModelPaneView+Power.swift"; sourceTree = ""; }; DFFEDED24AED136A9C83F730 /* SuggestionSettingsModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionSettingsModel.swift; sourceTree = ""; }; DFFF0D3BEE337265370C3AB0 /* EmojiPickerPanelLayoutTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiPickerPanelLayoutTests.swift; sourceTree = ""; }; E286B9A912808088FDA6B4C4 /* PermissionOverlayTracker.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PermissionOverlayTracker.swift; sourceTree = ""; }; + E2E6128CA72E5FE4BA856229 /* MLXHealthProbe.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXHealthProbe.swift; sourceTree = ""; }; E2F60BE5EDABE8882CA2F63B /* ConfidenceSuppressionPolicyTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConfidenceSuppressionPolicyTests.swift; sourceTree = ""; }; E34A14D9D852BB9C6829DDC7 /* DateMacroEvaluatorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DateMacroEvaluatorTests.swift; sourceTree = ""; }; E35DBAB63F24041B8A30E56A /* StreamedGhostTextPolicyTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = StreamedGhostTextPolicyTests.swift; sourceTree = ""; }; @@ -1621,8 +1700,10 @@ E45161160AC94229A630FC3A /* LlamaRuntimeManager.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LlamaRuntimeManager.swift; sourceTree = ""; }; E5B7FAD5A5E141AD9926D4A8 /* SystemResourceSampler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SystemResourceSampler.swift; sourceTree = ""; }; E5F6D44A9B4EC51505A0A363 /* KeybindRow.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KeybindRow.swift; sourceTree = ""; }; + E61D4E8C13EAD5806226C2C8 /* MLXModelDownloaderTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXModelDownloaderTests.swift; sourceTree = ""; }; E6DC2CCFDB2BF1F1E1375620 /* TerminalAppDetector.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TerminalAppDetector.swift; sourceTree = ""; }; E80071E4A5D844CEF5B7C487 /* RequestIDTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RequestIDTests.swift; sourceTree = ""; }; + E83059EDBC917FDE4F7AFE44 /* MLXCacheScanner.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXCacheScanner.swift; sourceTree = ""; }; E83C6E965A3978D0EBFB1EA2 /* ShortcutResolver.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ShortcutResolver.swift; sourceTree = ""; }; E8A00E909E0A51B119F9CB6B /* TypingHistoryPhraseEngineTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TypingHistoryPhraseEngineTests.swift; sourceTree = ""; }; E94A6036F4B703A2FF7F68EF /* DownloadFileRescuerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DownloadFileRescuerTests.swift; sourceTree = ""; }; @@ -1648,8 +1729,10 @@ EEDA42939A0406D27EB799A9 /* HostZoomLadderResolver.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HostZoomLadderResolver.swift; sourceTree = ""; }; EF21767F255146EA968B267D /* InlineFeatureTestDoubles.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InlineFeatureTestDoubles.swift; sourceTree = ""; }; EF4D6CEDF032A25EF72D7228 /* KeyCodeLabels.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KeyCodeLabels.swift; sourceTree = ""; }; + EF6D7EF81BDF13096889BED1 /* MLXModelDownloader.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXModelDownloader.swift; sourceTree = ""; }; F000304686E4573BC351AD3F /* PermissionDragSourceView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PermissionDragSourceView.swift; sourceTree = ""; }; F0A61DC16ED76D694A59BF14 /* SuggestionSessionTypingTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionSessionTypingTests.swift; sourceTree = ""; }; + F0DD6FD27894A80647739869 /* MLXLiveEvalTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MLXLiveEvalTests.swift; sourceTree = ""; }; F155556C751FB074B31CCC64 /* WelcomeHeroDemo.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = WelcomeHeroDemo.swift; sourceTree = ""; }; F1A4C5DEA32868F99E953D99 /* CompletionSeamGuard.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CompletionSeamGuard.swift; sourceTree = ""; }; F1B95D6456663C72DE3A6AF1 /* SuggestionEngineSelectionPolicyTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionEngineSelectionPolicyTests.swift; sourceTree = ""; }; @@ -1967,6 +2050,19 @@ path = Settings; sourceTree = ""; }; + 23EE8BECBE7EE6B6E7717AC6 /* MLX */ = { + isa = PBXGroup; + children = ( + 36C581E9E992813892959DBE /* MLXEngineWiringTests.swift */, + F0DD6FD27894A80647739869 /* MLXLiveEvalTests.swift */, + E61D4E8C13EAD5806226C2C8 /* MLXModelDownloaderTests.swift */, + AB4A82DFAD71F7568E801FB8 /* MLXProcessRunnerTests.swift */, + 2289DB842350106360F00DB9 /* MLXServerManagerTests.swift */, + 8617EB6F68EF450F58DFD59B /* MLXToolchainProvisionerTests.swift */, + ); + path = MLX; + sourceTree = ""; + }; 251BA8EDFCC71DCE20998F99 /* Emoji */ = { isa = PBXGroup; children = ( @@ -2210,6 +2306,7 @@ children = ( 6A7DD748979FC5BF6D0795A9 /* Metrics */, 7D2E806C1915514651343055 /* HuggingFaceModelsTests.swift */, + 803C81D876465AAAD03691AB /* MLXModelsTests.swift */, C40BA54CDA6F5887BEC6C668 /* OpenAICompatibleEndpointConfigurationTests.swift */, D72CC98CFDA795F8210198E8 /* RuntimeBootstrapModelTests.swift */, ); @@ -2382,6 +2479,7 @@ 3126B5AB15F52461DA9B5A00 /* Request */, F5B27DA75BE882A7DEB7D975 /* Session */, 4BA1D28712B262ABD96F8399 /* ModelAndPresentationValueTests.swift */, + 7B70ED445F6B46B9FE4176A5 /* SuggestionEngineKindTests.swift */, B494C76056D9B01D6A3ECABD /* SuggestionEngineModelsTests.swift */, CC5CD24D2FF35FD36B8D78DE /* SuggestionQualityMetricsStoreTests.swift */, ); @@ -2470,6 +2568,7 @@ 6A7DD748979FC5BF6D0795A9 /* Metrics */ = { isa = PBXGroup; children = ( + 493009C5372CBBA5A6C9E171 /* MenuResourceMonitorTests.swift */, BA991392A72658B758551BBA /* PerformanceMetricsStoreTests.swift */, 6A25AC9344DBA93EFA90788D /* SystemMetricsStoreTests.swift */, ); @@ -2525,6 +2624,7 @@ 61DFA08E857DB36A7876FFEA /* EngineAndModelPaneView+Actions.swift */, 39B891BA80475B61900A1D3C /* EngineAndModelPaneView+AppleIntelligence.swift */, 0A184538FD926947EBB88D9E /* EngineAndModelPaneView+Endpoint.swift */, + 057A4153C112521EDCFD1CC2 /* EngineAndModelPaneView+MLX.swift */, 8B2C3A2CEC6E0A13E2102D15 /* EngineAndModelPaneView+OpenSource.swift */, DF5872EC7795CC1EFF6D0D04 /* EngineAndModelPaneView+Power.swift */, ); @@ -2607,6 +2707,7 @@ children = ( F3CE75B5DCAA97579E453CCD /* AppleIntelligence */, D55AC5A7C15AD5E2917B168A /* Llama */, + 23EE8BECBE7EE6B6E7717AC6 /* MLX */, 3EFC2B112178CA99D6D76388 /* OpenAICompatible */, 6C034C1C9E51031A845D1184 /* SuggestionEngineRouterTests.swift */, E8A00E909E0A51B119F9CB6B /* TypingHistoryPhraseEngineTests.swift */, @@ -2674,6 +2775,20 @@ path = Context; sourceTree = ""; }; + 80BDFD25C2D4506F9ECBCA34 /* MLX */ = { + isa = PBXGroup; + children = ( + E2E6128CA72E5FE4BA856229 /* MLXHealthProbe.swift */, + EF6D7EF81BDF13096889BED1 /* MLXModelDownloader.swift */, + DECAC67D186C067A6F75517D /* MLXProcessRunner.swift */, + 578A470C822BB93EEC442960 /* MLXServerLogWriter.swift */, + 33291D41B4D72725738C3562 /* MLXServerManager.swift */, + 9093E08339428C7A0107D993 /* MLXToolchainInstaller.swift */, + 9CFBBF67EDCC6D7E19FFD04C /* MLXToolchainProvisioner.swift */, + ); + path = MLX; + sourceTree = ""; + }; 81A8C1B4295E6426050B3C2B /* Onboarding */ = { isa = PBXGroup; children = ( @@ -2710,6 +2825,7 @@ 14D5E77EB08030489DAFA4DC /* DeviceInfoTests.swift */, D08DF3F7E58429007C599247 /* GPUStatisticsReaderTests.swift */, 8A644C2ADAEC89E7AEE23A61 /* HardwareCapabilityProbeTests.swift */, + 7E348E1CB10B4EAAC9CA1572 /* ResourceUsageSummaryTests.swift */, B3FA4E40E8DF352247E1374B /* SystemResourceSamplerTests.swift */, ); path = Hardware; @@ -2799,6 +2915,7 @@ 6586D6D735F287F43D42520A /* DeviceInfo.swift */, EBB8E9D17613D574FC7B29F0 /* GPUStatisticsReader.swift */, 9F63A5EF992329FB9F6C4C26 /* HardwareCapabilityProbe.swift */, + 110BCAD18AFB2D446E5A607D /* ResourceUsageSummary.swift */, E5B7FAD5A5E141AD9926D4A8 /* SystemResourceSampler.swift */, ); path = Hardware; @@ -2852,6 +2969,19 @@ path = Evaluators; sourceTree = ""; }; + 97D36BF367122CDA8B65B0F5 /* MLX */ = { + isa = PBXGroup; + children = ( + E83059EDBC917FDE4F7AFE44 /* MLXCacheScanner.swift */, + 311D0D0F37A113911CAC1D07 /* MLXGeneratorFailure.swift */, + C659AC4B22F9E2E0D69D137E /* MLXModelCompatibility.swift */, + DF574B2D5A31E00ECE381724 /* MLXServerDesireResolver.swift */, + 044F1B6638860E6E8C37B5C6 /* MLXServerLogFilter.swift */, + 65674023B2B87D90B0B7878E /* MLXToolchainRules.swift */, + ); + path = MLX; + sourceTree = ""; + }; 987BC94D9D28389C8CE0406E /* Visual */ = { isa = PBXGroup; children = ( @@ -2941,6 +3071,7 @@ isa = PBXGroup; children = ( 920B8D1D67715E13372BB1FD /* Hardware */, + 97D36BF367122CDA8B65B0F5 /* MLX */, A29EA4138BBF767BA8D27223 /* Aria2Locator.swift */, 83457CCF1A50CE83428C363D /* Aria2OutputParser.swift */, CAA0ED5F9A319942CA024F93 /* Aria2StagingDirectoryResolver.swift */, @@ -3038,6 +3169,7 @@ BDD8D165BC46AFEC78AFAD13 /* Metrics */, C736C539D4C97B5664E498BE /* HuggingFaceModels.swift */, 16967F782C95CEB09B9C8D49 /* LlamaRuntimeModels.swift */, + 70F9C7C2FD21D85CBFFE4807 /* MLXModels.swift */, 1ABCE733783332BE52E43D67 /* OpenAICompatibleEndpointModels.swift */, 1DF8AA184C5F9CFFDF97A070 /* RuntimeBootstrapModel.swift */, ); @@ -3196,6 +3328,7 @@ BDD8D165BC46AFEC78AFAD13 /* Metrics */ = { isa = PBXGroup; children = ( + 1BC7B3FAE8ABAFE9910AEED2 /* MenuResourceMonitor.swift */, 024A80AEB32CC1AE8C0FF69E /* PerformanceMetricsStore.swift */, 1B966EA885CB107161FC3006 /* SystemMetricsStore.swift */, ); @@ -3274,6 +3407,18 @@ path = Request; sourceTree = ""; }; + C25B834FFB598A09F8BB2395 /* MLX */ = { + isa = PBXGroup; + children = ( + 9150E67CDE6F71584C8DFAA4 /* MLXCacheScannerTests.swift */, + CC8B01E696534BACF09ED6C7 /* MLXModelCompatibilityTests.swift */, + 27049BD6A6B9E46DE07E01C8 /* MLXServerDesireResolverTests.swift */, + 7ECD66E36EB8314C5A11A795 /* MLXServerLogFilterTests.swift */, + 8B87FBEAF425DEE74F008EE5 /* MLXToolchainRulesTests.swift */, + ); + path = MLX; + sourceTree = ""; + }; C28CA64D827F41D5C2C2E593 /* Power */ = { isa = PBXGroup; children = ( @@ -3557,6 +3702,7 @@ A306C7B2B0790A0B98D8F353 /* PerDomainDisableSettingsTests.swift */, DA02598F555696782E180DA5 /* ShortcutConflictTests.swift */, 2E7C883990AE384602BA9AF6 /* SuggestionSettingsDomainTests.swift */, + 93208561AC9A2CEB4F18D6C0 /* SuggestionSettingsMLXTests.swift */, CC30D79131D562C5CD7FA639 /* SuggestionSettingsModelDoubleTapTests.swift */, D253D06D941F60EDAF112625 /* SuggestionSettingsModelTests.swift */, ); @@ -3674,6 +3820,7 @@ isa = PBXGroup; children = ( 83E745F2135265B51DA3D57D /* Hardware */, + C25B834FFB598A09F8BB2395 /* MLX */, 924714B1688E884D9AB2A79D /* Aria2LocatorTests.swift */, 4189E399B4D9114816EEE93D /* Aria2OutputParserTests.swift */, 5C321373893D44FACD9471A3 /* Aria2StagingDirectoryResolverTests.swift */, @@ -3750,6 +3897,7 @@ children = ( 9DE7214E640C6F55297A245B /* AppleIntelligence */, F12953939DDAE88E16FF7509 /* Llama */, + 80BDFD25C2D4506F9ECBCA34 /* MLX */, D1FD19C28ED1EA45A68ABE16 /* OpenAICompatible */, 2C8D50115291212D03414CBE /* DebugForcedSuggestionEngine.swift */, 384FBCF5D7A3A446C5BE2B8D /* SuggestionEngineRouter.swift */, @@ -4061,6 +4209,7 @@ 6B1CD70508F906A62ABDBEDE /* EngineAndModelPaneView+Actions.swift in Sources */, E92B21A482D102860EE08EE8 /* EngineAndModelPaneView+AppleIntelligence.swift in Sources */, 9B7FE4C9ED6959A6D5181EF5 /* EngineAndModelPaneView+Endpoint.swift in Sources */, + 0D110609FA275045E66406A5 /* EngineAndModelPaneView+MLX.swift in Sources */, 19EB4D1C2D18C0ADC70BAD29 /* EngineAndModelPaneView+OpenSource.swift in Sources */, C6F2116D9256622BA879E2EE /* EngineAndModelPaneView+Power.swift in Sources */, E6650273045E6F1976AF23CA /* EngineAndModelPaneView.swift in Sources */, @@ -4134,6 +4283,20 @@ 3A5BA316620E502DAED81EE1 /* LlamaSuggestionEngine.swift in Sources */, B5241C6026A756A7FDE4F041 /* LocalRuntimeResidencyPolicy.swift in Sources */, C6867B979A03C7928BC83966 /* LowPowerModeMonitor.swift in Sources */, + 432853D9888B652846A81D8F /* MLXCacheScanner.swift in Sources */, + 34D97C80628E0085F151EFE7 /* MLXGeneratorFailure.swift in Sources */, + 9B59E38D1A606272C121EB51 /* MLXHealthProbe.swift in Sources */, + D8FC1912B9806CC1797397A0 /* MLXModelCompatibility.swift in Sources */, + FC9590FCC5F594D52836F78B /* MLXModelDownloader.swift in Sources */, + B39FB21A6FA1A214386EB433 /* MLXModels.swift in Sources */, + 5D5546A84BF1660CB2CFE9ED /* MLXProcessRunner.swift in Sources */, + 88FEE83F61272CCB04D1FC83 /* MLXServerDesireResolver.swift in Sources */, + 2A652B3FF4213C2454E1814B /* MLXServerLogFilter.swift in Sources */, + 3DBD5276426E05403FDCEB67 /* MLXServerLogWriter.swift in Sources */, + C20A1703F0DB96E48921C624 /* MLXServerManager.swift in Sources */, + 5424621834130DA5BAC8B432 /* MLXToolchainInstaller.swift in Sources */, + 99FF57753B69D792A3B4B4B6 /* MLXToolchainProvisioner.swift in Sources */, + AC376494710383408CB92DEB /* MLXToolchainRules.swift in Sources */, 647F30C40CE57EDFDA95958F /* MacroController.swift in Sources */, 1D54C941ED086C6427CFD773 /* MacroEngine.swift in Sources */, A614AD79BE724ABA3721613B /* MacroModels.swift in Sources */, @@ -4147,6 +4310,7 @@ 66E7FB51A0DF5916ABBA15D0 /* MenuBarSections.swift in Sources */, 4D4F02A7187C324A4B43A974 /* MenuBarStatusLabelView.swift in Sources */, 35F1681148A324CBEC797BAF /* MenuBarView.swift in Sources */, + 76C901B3BB16E0BEBC741C43 /* MenuResourceMonitor.swift in Sources */, 8D83B47447409074AAD1A423 /* MidWordContinuationPolicy.swift in Sources */, 222A03738679FC530E5B19B5 /* MirrorOverlayLayout.swift in Sources */, E6C99AB3E1A56E71F3A518FB /* ModelDownloadManager.swift in Sources */, @@ -4186,6 +4350,7 @@ 47EFFC4AEBA2848C933BB77C /* PromptSectionBudget.swift in Sources */, A6A0F90E7BCAAFE28A274DAF /* RandomMacroEvaluator.swift in Sources */, 3CF5E6D50BB799DFD6EBDA92 /* RequestID.swift in Sources */, + 851B43F86320BEBAF8B45EDC /* ResourceUsageSummary.swift in Sources */, 188DE63AA5C77C740781A1AF /* RuntimeBootstrapModel.swift in Sources */, 62B45766E8A2827E31AFD356 /* ScreenFrameReader.swift in Sources */, B6BA7DF77DCA55F9EF090AEE /* ScreenTextExtractor.swift in Sources */, @@ -4402,6 +4567,7 @@ 604D3D3F1D8966C6A976746C /* EngineAndModelPaneView+Actions.swift in Sources */, 1F728AD7AED5908FAA087EA9 /* EngineAndModelPaneView+AppleIntelligence.swift in Sources */, 713DD4EE84CE1BE9ADEB8E45 /* EngineAndModelPaneView+Endpoint.swift in Sources */, + E105B69FB9E82E68F636760B /* EngineAndModelPaneView+MLX.swift in Sources */, F45DC37FC763FB932B9AB3FB /* EngineAndModelPaneView+OpenSource.swift in Sources */, FE4F4A0778E7D2ABC9EEC155 /* EngineAndModelPaneView+Power.swift in Sources */, 8B8EA667506EB67DBCA35D57 /* EngineAndModelPaneView.swift in Sources */, @@ -4475,6 +4641,20 @@ 3CEF8F66CBBD5D828135FF9E /* LlamaSuggestionEngine.swift in Sources */, 5E2DB725654F411FB808AB01 /* LocalRuntimeResidencyPolicy.swift in Sources */, 01D6FF389B0E14C5BFDE4FBB /* LowPowerModeMonitor.swift in Sources */, + B090611E3D979A6C75303851 /* MLXCacheScanner.swift in Sources */, + A2BB842F89BAEEEC91DED837 /* MLXGeneratorFailure.swift in Sources */, + 8D04ACCC0BDD08806612E029 /* MLXHealthProbe.swift in Sources */, + 94D0A4675205578C85A1561E /* MLXModelCompatibility.swift in Sources */, + BF0BD554ACCEE680CACEC108 /* MLXModelDownloader.swift in Sources */, + BD9C9BDD38463AFAFE89EE6C /* MLXModels.swift in Sources */, + 0ED3FD222E15951E0FCAC9ED /* MLXProcessRunner.swift in Sources */, + B7EF26AC03B17D245B27F616 /* MLXServerDesireResolver.swift in Sources */, + 8DB6D558C9696ED2D562384B /* MLXServerLogFilter.swift in Sources */, + A2C7AE3FC56944880BE6B50A /* MLXServerLogWriter.swift in Sources */, + 8D1EB57587E5BB84758C6797 /* MLXServerManager.swift in Sources */, + F3F9E720ECF623E5A7F631CA /* MLXToolchainInstaller.swift in Sources */, + 351264BEFC3D30B585F480FD /* MLXToolchainProvisioner.swift in Sources */, + E7206CC66B0FDE5933BE2CF9 /* MLXToolchainRules.swift in Sources */, E04950B45EF6C6CE47EE5081 /* MacroController.swift in Sources */, 2EDFE6F33018D2D43D5CB813 /* MacroEngine.swift in Sources */, BBBD7B4628BA1160DD6B4BDB /* MacroModels.swift in Sources */, @@ -4488,6 +4668,7 @@ 832FB4EEA935CA4D67F4478D /* MenuBarSections.swift in Sources */, C52E6F9FECD2FB173386F836 /* MenuBarStatusLabelView.swift in Sources */, 7807D6217618D982CD301875 /* MenuBarView.swift in Sources */, + 1064D280B1851263DA3FE745 /* MenuResourceMonitor.swift in Sources */, 800D96A3889A915779826D11 /* MidWordContinuationPolicy.swift in Sources */, 2CF9087AE6FDB2A182BA9C85 /* MirrorOverlayLayout.swift in Sources */, 0CF057842A15F79FB3C69462 /* ModelDownloadManager.swift in Sources */, @@ -4527,6 +4708,7 @@ 21187878C7390AD639717B8F /* PromptSectionBudget.swift in Sources */, 4115E3AB3417BFA8DE43E6FD /* RandomMacroEvaluator.swift in Sources */, EA7D6ECD050AC621B02E35F2 /* RequestID.swift in Sources */, + C0E2C78678E89DEEBA7C50F1 /* ResourceUsageSummary.swift in Sources */, 0144ECDA06A8BDEFBB8DACE3 /* RuntimeBootstrapModel.swift in Sources */, 0A2D9EC761FDAEF15F547C4E /* ScreenFrameReader.swift in Sources */, 0C06CAD62975E87B2C852191 /* ScreenTextExtractor.swift in Sources */, @@ -4782,12 +4964,25 @@ 67C36958132793C76843F875 /* LlamaTypingSessionEvalTests.swift in Sources */, E79834E768C2001098CE6132 /* LocalRuntimeResidencyPolicyTests.swift in Sources */, C4ED7301D4DC86DC558B0C04 /* LowPowerModeMonitorTests.swift in Sources */, + B589039CB4C39E49C30E8816 /* MLXCacheScannerTests.swift in Sources */, + 1727CCFF2E46A8E471AE78B9 /* MLXEngineWiringTests.swift in Sources */, + B72EC0CDA91CD50EA401688C /* MLXLiveEvalTests.swift in Sources */, + 9153232EF26A3A9B10C8E62D /* MLXModelCompatibilityTests.swift in Sources */, + A06F4F2F50F4965811921312 /* MLXModelDownloaderTests.swift in Sources */, + 56EEAD59C2764ACDDFA3E597 /* MLXModelsTests.swift in Sources */, + 0787F247FDEE0287593770D7 /* MLXProcessRunnerTests.swift in Sources */, + 1A789303EB793105611E31AF /* MLXServerDesireResolverTests.swift in Sources */, + F3E29226CEF2792782E558B3 /* MLXServerLogFilterTests.swift in Sources */, + 7D420A402011BE5BB935700C /* MLXServerManagerTests.swift in Sources */, + E606A8F90DD5A91CE72581AC /* MLXToolchainProvisionerTests.swift in Sources */, + 3F7F2CB339DB6B1628A4E861 /* MLXToolchainRulesTests.swift in Sources */, 332F2AE6277FB6E871589E4D /* MacroControllerTests.swift in Sources */, AB8F186A7CB13779DAF729A0 /* MacroEngineTests.swift in Sources */, 8756E3050C45106D7AD188AA /* MacroTriggerStateMachineTests.swift in Sources */, 0C89AF5CF2114DF57DA794CF /* MailHeaderFieldDetectorTests.swift in Sources */, 1B80604D77F1AF1DF96DDD45 /* MarkerSelectionSynthesizerTests.swift in Sources */, A54337B0B12D1FE2F332C7E1 /* MenuBarRecoveryPolicyTests.swift in Sources */, + CF0FCA57D1CBC6379147960E /* MenuResourceMonitorTests.swift in Sources */, 85BB2FAD4BFE9077AEE2A262 /* MidWordContinuationPolicyTests.swift in Sources */, 081EB4DF65AD5736DE7FBFDE /* MirrorOverlayLayoutTests.swift in Sources */, B3E9C077CF064F2DEC3E22B8 /* ModelAndPresentationValueTests.swift in Sources */, @@ -4823,6 +5018,7 @@ EC4975F707689CEA77A636BB /* PromptSectionBudgetTests.swift in Sources */, 5E506F0CD6FA82B323264C88 /* RandomMacroEvaluatorTests.swift in Sources */, 27B1A99D61398752863DB251 /* RequestIDTests.swift in Sources */, + 098F4520E2D5BEA2CE45C4A1 /* ResourceUsageSummaryTests.swift in Sources */, FAEC5301B36B92E4B4A169D3 /* RuntimeBootstrapModelTests.swift in Sources */, 03FBB478CF7C51BDFB9AEB6D /* ScreenTextExtractorTests.swift in Sources */, 050750A0343235F426D8B47B /* ScreenshotContextGeneratorTests.swift in Sources */, @@ -4862,6 +5058,7 @@ A1712224D1E7273EDA047544 /* SuggestionCoordinatorWordCompletionTests.swift in Sources */, D98538D3C1344422354B0B2E /* SuggestionDebugLoggerTests.swift in Sources */, 85EDFFAB94CB8577CDBAF49B /* SuggestionDismissalMemoryTests.swift in Sources */, + 75845759F2066E0558AFDFC7 /* SuggestionEngineKindTests.swift in Sources */, 3A3B49975BCF1F648F8C1782 /* SuggestionEngineModelsTests.swift in Sources */, F76ECB9663E46AA806E2C7DE /* SuggestionEngineRouterTests.swift in Sources */, 7D4D95E3626A1263DF8A0AC4 /* SuggestionEngineSelectionPolicyTests.swift in Sources */, @@ -4880,6 +5077,7 @@ 76B2F9D2C93958F10F65D26F /* SuggestionSessionReconciliationTests.swift in Sources */, B8B71602DFA0E6B7E61E2692 /* SuggestionSessionTypingTests.swift in Sources */, E83E989C5132F2A8C8FED8EB /* SuggestionSettingsDomainTests.swift in Sources */, + F0F06AF3A0508BE380AD3CAD /* SuggestionSettingsMLXTests.swift in Sources */, 6471AEB3C75714538D78DBA3 /* SuggestionSettingsModelDisabledAppsTests.swift in Sources */, EA16F79329E37D921F0BB542 /* SuggestionSettingsModelDoubleTapTests.swift in Sources */, 2D83CECB0FF076323226449D /* SuggestionSettingsModelTests.swift in Sources */, diff --git a/Cotabby/App/Coordinators/SettingsCoordinator.swift b/Cotabby/App/Coordinators/SettingsCoordinator.swift index 77a288b49..31d1c494d 100644 --- a/Cotabby/App/Coordinators/SettingsCoordinator.swift +++ b/Cotabby/App/Coordinators/SettingsCoordinator.swift @@ -16,6 +16,9 @@ final class SettingsCoordinator: NSObject, NSWindowDelegate { private let permissionGuidanceController: PermissionGuidanceController private let suggestionSettings: SuggestionSettingsModel private let openAICompatibleConnectionModel: OpenAICompatibleConnectionModel + private let mlxToolchainProvisioner: MLXToolchainProvisioner + private let mlxServerManager: MLXServerManager + private let mlxModelDownloader: MLXModelDownloader private let foundationModelAvailabilityService: FoundationModelAvailabilityService private let runtimeModel: RuntimeBootstrapModel private let modelDownloadManager: ModelDownloadManager @@ -42,6 +45,9 @@ final class SettingsCoordinator: NSObject, NSWindowDelegate { permissionGuidanceController: PermissionGuidanceController, suggestionSettings: SuggestionSettingsModel, openAICompatibleConnectionModel: OpenAICompatibleConnectionModel, + mlxToolchainProvisioner: MLXToolchainProvisioner, + mlxServerManager: MLXServerManager, + mlxModelDownloader: MLXModelDownloader, foundationModelAvailabilityService: FoundationModelAvailabilityService, runtimeModel: RuntimeBootstrapModel, modelDownloadManager: ModelDownloadManager, @@ -58,6 +64,9 @@ final class SettingsCoordinator: NSObject, NSWindowDelegate { self.permissionGuidanceController = permissionGuidanceController self.suggestionSettings = suggestionSettings self.openAICompatibleConnectionModel = openAICompatibleConnectionModel + self.mlxToolchainProvisioner = mlxToolchainProvisioner + self.mlxServerManager = mlxServerManager + self.mlxModelDownloader = mlxModelDownloader self.foundationModelAvailabilityService = foundationModelAvailabilityService self.runtimeModel = runtimeModel self.modelDownloadManager = modelDownloadManager @@ -88,6 +97,9 @@ final class SettingsCoordinator: NSObject, NSWindowDelegate { permissionGuidanceController: permissionGuidanceController, suggestionSettings: suggestionSettings, openAICompatibleConnectionModel: openAICompatibleConnectionModel, + mlxToolchainProvisioner: mlxToolchainProvisioner, + mlxServerManager: mlxServerManager, + mlxModelDownloader: mlxModelDownloader, foundationModelAvailabilityService: foundationModelAvailabilityService, runtimeModel: runtimeModel, modelDownloadManager: modelDownloadManager, diff --git a/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+Continuation.swift b/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+Continuation.swift index 7660123e4..2cd893e1c 100644 --- a/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+Continuation.swift +++ b/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+Continuation.swift @@ -32,10 +32,12 @@ extension SuggestionCoordinator { /// Only terminal word endings need another request. A model-provided phrase stays in its /// original session; prefetch never replaces those words with a different model guess. func prepareContinuation(after session: ActiveSuggestionSession, rawContext: FocusedInputSnapshot) { - // Uncommitted lookahead is new work, so keep it on-device. The coordinator intentionally - // has no endpoint URL; allowing all configured endpoints here would also send hypothetical - // corrections to hosted servers. Their already-returned phrases still use the same buffer. - guard settingsSnapshot.selectedEngine != .openAICompatible, + // Uncommitted lookahead is new work the user did not ask for. Only the in-process engines + // yield it to the real request at once; on a server (the endpoint or MLX) it shares the + // generation work with the suggestion the user waits for until its cancellation arrives, + // and the endpoint would also receive hypothetical corrections. Already-returned phrases + // still use the same buffer. + guard settingsSnapshot.selectedEngine.allowsSpeculativeGeneration, !userDefaults.bool(forKey: Self.speculativePrefetchDisabledDefaultsKey), currentDisabledReason(focusSnapshot: focusModel.snapshot) == nil else { return } let plan: SuggestionContinuationPlan? diff --git a/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+TypingPrediction.swift b/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+TypingPrediction.swift index b56e49b25..3b68703ce 100644 --- a/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+TypingPrediction.swift +++ b/Cotabby/App/Coordinators/Suggestion/SuggestionCoordinator+TypingPrediction.swift @@ -6,10 +6,11 @@ import Foundation extension SuggestionCoordinator { func beginTypingPrediction(for request: SuggestionRequest) { clearTypingPrediction() - // Internal partial collection must not add speculative traffic to configured endpoints. + // Internal partial collection must not add speculative requests to a server (the endpoint + // or MLX), where they would share generation work with the request the user waits for. // Post-acceptance speculation has its own publication contract and is kept separate. guard settingsSnapshot.predictAheadWhileTyping, - settingsSnapshot.selectedEngine != .openAICompatible, + settingsSnapshot.selectedEngine.allowsSpeculativeGeneration, pendingSpeculativeContext == nil, !request.context.isSecure, request.context.selection.length == 0, !userDefaults.bool(forKey: Self.speculativePrefetchDisabledDefaultsKey) else { return } diff --git a/Cotabby/App/Core/AppDelegate.swift b/Cotabby/App/Core/AppDelegate.swift index 432b405b9..9a77be087 100644 --- a/Cotabby/App/Core/AppDelegate.swift +++ b/Cotabby/App/Core/AppDelegate.swift @@ -25,6 +25,8 @@ final class AppDelegate: NSObject, NSApplicationDelegate { let suggestionSettings: SuggestionSettingsModel let foundationModelAvailabilityService: FoundationModelAvailabilityService let powerSourceMonitor: PowerSourceMonitor + let mlxServerManager: MLXServerManager + let menuResourceMonitor: MenuResourceMonitor let suggestionCoordinator: SuggestionCoordinator let inlineCommandCoordinator: InlineCommandCoordinator let welcomeCoordinator: WelcomeCoordinator @@ -57,6 +59,8 @@ final class AppDelegate: NSObject, NSApplicationDelegate { suggestionSettings = environment.suggestionSettings foundationModelAvailabilityService = environment.foundationModelAvailabilityService powerSourceMonitor = environment.powerSourceMonitor + mlxServerManager = environment.mlxServerManager + menuResourceMonitor = environment.menuResourceMonitor suggestionCoordinator = environment.suggestionCoordinator inlineCommandCoordinator = environment.inlineCommandCoordinator welcomeCoordinator = environment.welcomeCoordinator @@ -164,6 +168,7 @@ final class AppDelegate: NSObject, NSApplicationDelegate { ) applyLaunchAtLoginDefaultIfNeeded() startRuntimeIfPreferredEngineRequiresIt() + environment.startMLXEngine() focusModel.start() inputMonitor.start() appUpdateManager.start() @@ -256,6 +261,9 @@ final class AppDelegate: NSObject, NSApplicationDelegate { inputMonitor.stop() focusModel.stop() + // Waits at most 3 s for mlx_lm.server to exit (SIGKILL after that), so quitting Cotabby + // never leaves a Python process holding the model's memory. + environment.mlxServerManager.shutdownSync(timeoutSeconds: 3) runtimeModel.shutdownSync(timeoutSeconds: 1.5) } @@ -296,8 +304,9 @@ final class AppDelegate: NSObject, NSApplicationDelegate { runtimeModel.startIfNeeded() } else { // Switching away must release Metal buffers and the mapped GGUF. Otherwise an Ollama - // user still pays the duplicate memory cost the external endpoint is meant to avoid, and - // an Apple Intelligence user keeps a model they asked not to keep loaded. + // user still pays the duplicate memory cost the external endpoint is meant to avoid, an + // Apple Intelligence user keeps a model they asked not to keep loaded, and an MLX user + // holds a second copy of the weights beside the one mlx_lm.server loaded. runtimeModel.stop() } } diff --git a/Cotabby/App/Core/CotabbyApp.swift b/Cotabby/App/Core/CotabbyApp.swift index ee1002902..815b43dd7 100644 --- a/Cotabby/App/Core/CotabbyApp.swift +++ b/Cotabby/App/Core/CotabbyApp.swift @@ -27,6 +27,8 @@ struct CotabbyApp: App { suggestionSettings: appDelegate.suggestionSettings, foundationModelAvailabilityService: appDelegate.foundationModelAvailabilityService, powerSourceMonitor: appDelegate.powerSourceMonitor, + mlxServerManager: appDelegate.mlxServerManager, + menuResourceMonitor: appDelegate.menuResourceMonitor, appUpdateManager: appDelegate.appUpdateManager, onOpenSettings: { appDelegate.settingsCoordinator.showSettings() diff --git a/Cotabby/App/Core/CotabbyAppEnvironment.swift b/Cotabby/App/Core/CotabbyAppEnvironment.swift index 2b8b97338..c1f872f85 100644 --- a/Cotabby/App/Core/CotabbyAppEnvironment.swift +++ b/Cotabby/App/Core/CotabbyAppEnvironment.swift @@ -22,6 +22,12 @@ final class CotabbyAppEnvironment { let permissionGuidanceController: PermissionGuidanceController let suggestionSettings: SuggestionSettingsModel let openAICompatibleConnectionModel: OpenAICompatibleConnectionModel + /// MLX engine services, created once and kept for the app's lifetime. Settings observes them; + /// `startMLXEngine()` (called by AppDelegate outside the test host) starts their work. + let mlxToolchainProvisioner: MLXToolchainProvisioner + let mlxServerManager: MLXServerManager + let menuResourceMonitor: MenuResourceMonitor + let mlxModelDownloader: MLXModelDownloader let foundationModelAvailabilityService: FoundationModelAvailabilityService let powerSourceMonitor: PowerSourceMonitor /// Process-lifetime Low Power Mode observer shared with suggestion orchestration. @@ -46,6 +52,9 @@ final class CotabbyAppEnvironment { let focusDebugOverlayController: FocusDebugOverlayController? private var cancellables = Set() + /// Serializes `apply(_:)` calls to the MLX server manager: each new task awaits the previous + /// one, so a quick engine switch and switch-back reach the manager in order. + private var mlxApplyChain: Task? init() { CotabbyLogger.app.info("Building dependency graph") @@ -66,6 +75,10 @@ final class CotabbyAppEnvironment { let openAICompatibleConnectionModel = OpenAICompatibleConnectionModel( client: openAICompatibleClient ) + let mlxPaths = MLXPaths.standard() + let mlxToolchainProvisioner = MLXToolchainProvisioner.live(paths: mlxPaths) + let mlxServerManager = MLXServerManager.live(paths: mlxPaths) + let mlxModelDownloader = MLXModelDownloader.live() let foundationModelAvailabilityService = FoundationModelAvailabilityService() let powerSourceMonitor = PowerSourceMonitor() let lowPowerModeMonitor = LowPowerModeMonitor() @@ -156,7 +169,24 @@ final class CotabbyAppEnvironment { let qualityMetricsStore = SuggestionQualityMetricsStore() // Live CPU/RAM graph backing for the Performance pane. Holds no state until the pane asks it // to start sampling, so constructing it eagerly here costs nothing. - let systemMetricsStore = SystemMetricsStore() + // With MLX the model runs in a separate server process, so the sampler adds that process's + // CPU and memory while one is running; otherwise the graph would read near zero. + let systemMetricsStore = SystemMetricsStore(sampler: { [weak mlxServerManager] in + let pid = MainActor.assumeIsolated { mlxServerManager?.runningProcessIdentifier } + var sample = SystemResourceSampler.sample(extraProcessIDs: pid.map { [$0] } ?? []) + sample.includesMLXServer = pid != nil + return sample + }) + // Compact CPU/RAM/GPU line in the menu bar popover. It samples with its own baselines + // (`MenuResourceSampler`) so it cannot disturb the Performance pane's CPU deltas. + let menuResourceSampler = MenuResourceSampler() + let menuResourceMonitor = MenuResourceMonitor( + sample: { [weak mlxServerManager] in + let pid = MainActor.assumeIsolated { mlxServerManager?.runningProcessIdentifier } + return menuResourceSampler.sample(extraProcessID: pid) + }, + onStop: { menuResourceSampler.reset() } + ) let suggestionInserter = SuggestionInserter(suppressionController: suppressionController) // Commit accepted text through an IME-safe path (Accessibility / paste) while a composing IME // is active; a synthetic keystroke would be re-absorbed into composition and the accept would @@ -220,6 +250,13 @@ final class CotabbyAppEnvironment { ), endpointModelNameProvider: { [weak suggestionSettings] in suggestionSettings?.openAICompatibleModelName.nonEmpty + }, + mlxEngine: OpenAICompatibleSuggestionEngine.mlx( + client: openAICompatibleClient, + serverManager: mlxServerManager + ), + mlxModelNameProvider: { [weak suggestionSettings] in + suggestionSettings?.mlxModelID.nonEmpty } ) // Under `-cotabby-debug` with `cotabbyDebugForcedSuggestion` set, every request answers with @@ -249,6 +286,9 @@ final class CotabbyAppEnvironment { permissionGuidanceController: permissionGuidanceController, suggestionSettings: suggestionSettings, openAICompatibleConnectionModel: openAICompatibleConnectionModel, + mlxToolchainProvisioner: mlxToolchainProvisioner, + mlxServerManager: mlxServerManager, + mlxModelDownloader: mlxModelDownloader, foundationModelAvailabilityService: foundationModelAvailabilityService, runtimeModel: runtimeModel, modelDownloadManager: modelDownloadManager, @@ -352,6 +392,10 @@ final class CotabbyAppEnvironment { self.permissionGuidanceController = permissionGuidanceController self.suggestionSettings = suggestionSettings self.openAICompatibleConnectionModel = openAICompatibleConnectionModel + self.mlxToolchainProvisioner = mlxToolchainProvisioner + self.mlxServerManager = mlxServerManager + self.menuResourceMonitor = menuResourceMonitor + self.mlxModelDownloader = mlxModelDownloader self.foundationModelAvailabilityService = foundationModelAvailabilityService self.powerSourceMonitor = powerSourceMonitor self.lowPowerModeMonitor = lowPowerModeMonitor @@ -440,6 +484,8 @@ final class CotabbyAppEnvironment { suggestionSettings.$pluggedInEngine.map { _ in () }.eraseToAnyPublisher(), suggestionSettings.$pluggedInModelFilename.map { _ in () }.eraseToAnyPublisher(), suggestionSettings.$pluggedInEndpointModelName.map { _ in () }.eraseToAnyPublisher(), + suggestionSettings.$batteryMLXModelID.map { _ in () }.eraseToAnyPublisher(), + suggestionSettings.$pluggedInMLXModelID.map { _ in () }.eraseToAnyPublisher(), runtimeModel.$availableModels.map { _ in () }.eraseToAnyPublisher() ] @@ -500,6 +546,10 @@ final class CotabbyAppEnvironment { guard !modelName.isEmpty else { return } suggestionSettings.setOpenAICompatibleModelName(modelName) suggestionSettings.selectEngine(.openAICompatible) + case .mlx(let modelID): + guard !modelID.isEmpty else { return } + suggestionSettings.setMLXModelID(modelID) + suggestionSettings.selectEngine(.mlx) } } @@ -540,6 +590,85 @@ final class CotabbyAppEnvironment { } .store(in: &cancellables) } + + /// Starts MLX housekeeping once services start; never inside the XCTest host, where a stored + /// MLX selection must not launch Python. Order matters: a server orphaned by an earlier crash + /// is stopped before the toolchain check and cache scan, whose results then drive the server. + func startMLXEngine() { + Task { [weak self] in + guard let self else { return } + await self.mlxServerManager.cleanUpOrphan() + await self.mlxToolchainProvisioner.rescan(models: self.mlxModelDownloader) + } + observeMLXEngine() + } + + private func observeMLXEngine() { + // The toolchain's Python decides which architectures count as usable, so rescan the + // cache whenever a different install becomes ready (or goes away). + mlxToolchainProvisioner.$state + .map(\.toolchain) + .removeDuplicates() + .dropFirst() + .sink { [weak self] toolchain in + guard let self else { return } + Task { await self.mlxModelDownloader.refreshCache(toolchain: toolchain) } + } + .store(in: &cancellables) + + // Switching to MLX re-detects, so a toolchain or model changed by another tool since + // launch is noticed before the desire is resolved from stale state. + suggestionSettings.$selectedEngine + .removeDuplicates() + .dropFirst() // the launch value is covered by startMLXEngine's own rescan + .filter { $0 == .mlx } + .sink { [weak self] _ in + guard let self else { return } + Task { await self.mlxToolchainProvisioner.rescan(models: self.mlxModelDownloader) } + } + .store(in: &cancellables) + + // Choosing MLX with no model picks a downloaded one, preferring the curated list. + Publishers.CombineLatest(suggestionSettings.$selectedEngine, mlxModelDownloader.$cachedModels) + .sink { [weak self] engine, cached in + guard let self, engine == .mlx, self.suggestionSettings.mlxModelID.isEmpty, + let cached, let pick = MLXModelCatalog.defaultSelection(cached: cached) + else { return } + self.suggestionSettings.setMLXModelID(pick.value) + } + .store(in: &cancellables) + + // One pure decision from every input; each change is applied in order, so a quick + // engine switch and switch-back cannot reach the manager out of order. + Publishers.CombineLatest4( + suggestionSettings.$selectedEngine, + suggestionSettings.$mlxModelID, + suggestionSettings.$mlxPromptCacheBytes, + Publishers.CombineLatest3( + mlxToolchainProvisioner.$state, mlxModelDownloader.$cachedModels, mlxModelDownloader.$rejectedModels) + ) + .map { engine, modelID, cacheBytes, toolchainAndCache in + MLXServerDesireResolver.desire(for: .init( + engine: engine, + isAppleSilicon: MLXHardware.isAppleSilicon, + toolchain: toolchainAndCache.0, + modelID: modelID, + cachedModels: toolchainAndCache.1, + promptCacheBytes: cacheBytes, + rejectedModels: toolchainAndCache.2 + )) + } + .removeDuplicates() + .sink { [weak self] desire in + guard let self else { return } + let previous = self.mlxApplyChain + self.mlxApplyChain = Task { [weak self] in + await previous?.value + await self?.mlxServerManager.apply(desire) + } + } + .store(in: &cancellables) + } } private extension String { diff --git a/Cotabby/Models/Context/VisualContextModels.swift b/Cotabby/Models/Context/VisualContextModels.swift index e21aaf9f4..131ee079b 100644 --- a/Cotabby/Models/Context/VisualContextModels.swift +++ b/Cotabby/Models/Context/VisualContextModels.swift @@ -30,7 +30,7 @@ nonisolated struct VisualContextConfiguration: Equatable, Sendable { ) static func forEngine(_ engine: SuggestionEngineKind) -> Self { - engine == .openAICompatible ? .default : .local + engine.isOnDevice ? .local : .default } static let `default` = VisualContextConfiguration( diff --git a/Cotabby/Models/Runtime/MLXModels.swift b/Cotabby/Models/Runtime/MLXModels.swift new file mode 100644 index 000000000..b407f4bc2 --- /dev/null +++ b/Cotabby/Models/Runtime/MLXModels.swift @@ -0,0 +1,368 @@ +import Foundation + +/// File overview: +/// Shared values for the MLX engine: what this Mac can run, where Cotabby keeps its MLX install, +/// which models it offers, and the states the toolchain, server and downloads publish. Settings, +/// the router and the MLX services all read these, so they live in `Models/` rather than inside +/// any one service. + +/// Whether this build can run MLX. MLX needs Apple Silicon. A compile-time check is enough: an +/// x86_64 Cotabby (Intel, or Rosetta) cannot rely on an arm64 Python stack being usable either. +nonisolated enum MLXHardware { + static let isAppleSilicon: Bool = { + #if arch(arm64) + return true + #else + return false + #endif + }() +} + +/// The sampling fields `mlx_lm.server` accepts beyond OpenAI's basics. The MLX engine fills them +/// from the same request values the llama runtime samples with, so both local engines condition +/// alike. The client encodes them only when present, so hosted endpoints never see keys they +/// might reject. +nonisolated struct MLXCompletionOptions: Equatable, Sendable { + let topK: Int + let minP: Double + let repetitionPenalty: Double + /// Nil lets the server pick; Cotabby passes the request's seed so a replay is reproducible. + let seed: Int? + /// Server-side stop strings. `["\n"]` for single-line fields stands in for the llama runtime's + /// newline-token masking; empty for multi-line fields. + let stop: [String] +} + +/// Where Cotabby keeps its own MLX install and server bookkeeping. The folder sits beside the +/// llama runtime folder under the bundle's own Application Support name, so the Dev and release +/// apps never share an install or a pid file. +nonisolated struct MLXPaths: Equatable, Sendable { + let root: URL + + init(root: URL) { + self.root = root + } + + /// `~/Library/Application Support//MLX`. The folder is keyed by `CFBundleName` + /// so the Dev and release apps (different names) get separate installs and cannot delete or + /// overwrite each other's toolchain. + static func standard(bundle: Bundle = .main) -> MLXPaths { + let support = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask).first + ?? URL(fileURLWithPath: NSHomeDirectory()).appendingPathComponent("Library/Application Support") + let appName = (bundle.object(forInfoDictionaryKey: "CFBundleName") as? String) ?? "Cotabby" + return MLXPaths(root: support.appendingPathComponent(appName, isDirectory: true) + .appendingPathComponent("MLX", isDirectory: true)) + } + + var uvDirectory: URL { root.appendingPathComponent("uv", isDirectory: true) } + /// The archive unpacks into a folder named after the platform triple. + var uvExecutable: URL { uvDirectory.appendingPathComponent("uv-aarch64-apple-darwin/uv") } + var toolsDirectory: URL { root.appendingPathComponent("tools", isDirectory: true) } + var binDirectory: URL { root.appendingPathComponent("bin", isDirectory: true) } + var pythonDirectory: URL { root.appendingPathComponent("python", isDirectory: true) } + var cacheDirectory: URL { root.appendingPathComponent("cache", isDirectory: true) } + var ownServerExecutable: URL { binDirectory.appendingPathComponent("mlx_lm.server") } + var serverPIDFile: URL { root.appendingPathComponent("server.pid") } +} + +/// A validated Hugging Face model repo id ("owner/name"). It reaches a command line +/// (`--model`, `snapshot_download`) and a cache folder name, so only Hugging Face's own name +/// alphabet is accepted, never path syntax. A pasted model-page URL is reduced to its id. +nonisolated struct MLXRepoID: Equatable, Hashable, Sendable, CustomStringConvertible { + let owner: String + let name: String + + var value: String { "\(owner)/\(name)" } + var description: String { value } + /// The folder `huggingface_hub` caches this repo under. + var cacheFolderName: String { "models--\(owner)--\(name)" } + + /// First path segments that Hugging Face uses for its own pages (datasets, Spaces, listings), + /// so a URL starting with one of them is not a model page. + private static let reservedURLSegments: Set = [ + "api", "collections", "datasets", "docs", "models", "organizations", "papers", "settings", "spaces" + ] + + init?(_ raw: String) { + var text = raw.trimmingCharacters(in: .whitespacesAndNewlines) + // A link copied without its scheme has no host for `URL` to find; give it one. + let lowered = text.lowercased() + if lowered.hasPrefix("huggingface.co/") || lowered.hasPrefix("www.huggingface.co/") { + text = "https://" + text + } + if let url = URL(string: text), let host = url.host { + guard host == "huggingface.co" || host == "www.huggingface.co" else { return nil } + let parts = url.path.split(separator: "/").map(String.init) + guard parts.count >= 2, !Self.reservedURLSegments.contains(parts[0]) else { return nil } + text = "\(parts[0])/\(parts[1])" + } + let parts = text.split(separator: "/", omittingEmptySubsequences: false).map(String.init) + guard parts.count == 2, Self.isValidComponent(parts[0]), Self.isValidComponent(parts[1]) else { + return nil + } + owner = parts[0] + name = parts[1] + } + + init?(cacheFolderName: String) { + guard cacheFolderName.hasPrefix("models--") else { return nil } + let parts = cacheFolderName.dropFirst("models--".count).components(separatedBy: "--") + guard parts.count == 2 else { return nil } + self.init("\(parts[0])/\(parts[1])") + } + + /// Hugging Face allows letters, digits, `-`, `_` and `.`, forbids `--` and `..`, and names + /// cannot start with `-` or `.`; at most 96 characters. + private static func isValidComponent(_ text: String) -> Bool { + guard !text.isEmpty, text.count <= 96, + let first = text.unicodeScalars.first, + first != "-", first != ".", + !text.contains("--"), !text.contains("..") + else { return false } + let allowed = CharacterSet.alphanumerics.union(CharacterSet(charactersIn: "-_.")) + return text.unicodeScalars.allSatisfy { $0.isASCII && allowed.contains($0) } + } +} + +/// One model Cotabby recommends, shown even before it is downloaded. +nonisolated struct MLXCatalogEntry: Equatable, Sendable, Identifiable { + let repoID: MLXRepoID + let displayName: String + let approximateBytes: Int64 + let note: String + let isRecommended: Bool + var id: String { repoID.value } +} + +/// A usable MLX model found in the Hugging Face cache (produced by `MLXCacheScanner`). +nonisolated struct MLXCachedModel: Equatable, Sendable, Identifiable { + let repoID: MLXRepoID + let snapshotURL: URL + let sizeBytes: Int64 + let modelType: String + let isInstruct: Bool + var id: String { repoID.value } +} + +/// One row of the Settings model list: a curated or cached model, downloaded or not. +nonisolated struct MLXModelOption: Equatable, Sendable, Identifiable { + let repoID: MLXRepoID + let displayName: String + let sizeBytes: Int64 + let isDownloaded: Bool + let isRecommended: Bool + let note: String? + var id: String { repoID.value } +} + +/// The curated list and the rule that merges it with what the cache holds. A pure namespace: the +/// downloader publishes the cached models and the pane asks this type for the rows to show. +nonisolated enum MLXModelCatalog { + static let instructNote = "Instruct model: base models complete text better." + + static let curated: [MLXCatalogEntry] = [ + MLXCatalogEntry( + repoID: MLXRepoID("mlx-community/gemma-4-e4b-5bit")!, + displayName: "Gemma 4 E4B (5-bit)", + approximateBytes: 6_100_000_000, + note: "Recommended. Same model family as the Open Source default.", + isRecommended: true + ), + MLXCatalogEntry( + repoID: MLXRepoID("mlx-community/gemma-4-e2b-6bit")!, + displayName: "Gemma 4 E2B (6-bit)", + approximateBytes: 4_700_000_000, + note: "Smaller and faster.", + isRecommended: false + ) + ] + + /// Curated entries first, in catalog order, then every other cached model sorted by id. + static func options(cached: [MLXCachedModel], curated: [MLXCatalogEntry] = curated) -> [MLXModelOption] { + let cachedByID = Dictionary(cached.map { ($0.repoID, $0) }, uniquingKeysWith: { first, _ in first }) + let curatedIDs = Set(curated.map(\.repoID)) + let curatedRows = curated.map { entry in + let hit = cachedByID[entry.repoID] + return MLXModelOption( + repoID: entry.repoID, + displayName: entry.displayName, + sizeBytes: hit?.sizeBytes ?? entry.approximateBytes, + isDownloaded: hit != nil, + isRecommended: entry.isRecommended, + note: entry.note + ) + } + let otherRows = cached + .filter { !curatedIDs.contains($0.repoID) } + .sorted { $0.repoID.value.localizedCaseInsensitiveCompare($1.repoID.value) == .orderedAscending } + .map { model in + MLXModelOption( + repoID: model.repoID, + displayName: model.repoID.value, + sizeBytes: model.sizeBytes, + isDownloaded: true, + isRecommended: false, + note: model.isInstruct ? instructNote : nil + ) + } + return curatedRows + otherRows + } + + /// The download that no row in `options` shows: a custom model started from the "Other MLX + /// Model" field is not in the catalog until it is cached, so its progress, Cancel and failure + /// need a place of their own. Nil when the download (if any) already has a row. + static func downloadWithoutRow(_ download: MLXDownloadState?, options: [MLXModelOption]) -> MLXDownloadState? { + guard let download, !options.contains(where: { $0.repoID == download.repoID }) else { return nil } + return download + } + + /// The model to select when the user picks MLX without having chosen one: a downloaded + /// curated model first, then any downloaded model, else nil (the pane then offers downloads). + static func defaultSelection(cached: [MLXCachedModel]) -> MLXRepoID? { + let cachedIDs = Set(cached.map(\.repoID)) + if let curatedHit = curated.first(where: { cachedIDs.contains($0.repoID) }) { + return curatedHit.repoID + } + return cached.first?.repoID + } +} + +/// A usable mlx-lm install: the server script and the Python that runs it. +nonisolated struct MLXToolchain: Equatable, Sendable { + let serverExecutable: URL + let pythonExecutable: URL + let version: String + /// True for Cotabby's own install under `MLXPaths`, which Remove may delete. + let isCotabbyInstall: Bool +} + +/// The steps of Cotabby's own MLX install, in order. The installer reports the current one so the +/// Settings pane can say what is happening during a multi-minute wait; the provisioner publishes it +/// inside `MLXToolchainState.installing`. +nonisolated enum MLXInstallPhase: String, Equatable, Sendable { + case downloadingUV + case verifyingUV + case installingMLX + + var label: String { + switch self { + case .downloadingUV: return "Downloading the installer…" + case .verifyingUV: return "Verifying the installer…" + case .installingMLX: return "Installing MLX (a few minutes the first time)…" + } + } +} + +/// What the Settings pane and the server wiring know about mlx-lm on this Mac. +nonisolated enum MLXToolchainState: Equatable, Sendable { + case checking + case notInstalled + case tooOld(version: String, path: String) + case installing(MLXInstallPhase) + case ready(MLXToolchain) + case failed(reason: String, logTail: [String]) + + var toolchain: MLXToolchain? { + if case .ready(let toolchain) = self { return toolchain } + return nil + } +} + +/// Why the MLX server is not running although MLX is the selected engine. +nonisolated enum MLXUnavailableReason: Equatable, Sendable { + case unsupportedHardware + case notInstalled + case noModel + case modelNotDownloaded(MLXRepoID) + /// The model is downloaded but `MLXModelCompatibility` rejected it; the string is the reason. + case modelIncompatible(MLXRepoID, String) + + var message: String { + switch self { + case .unsupportedHardware: return "MLX needs a Mac with Apple Silicon." + case .notInstalled: return "Install MLX in Settings → Engine & Model." + case .noModel: return "Choose an MLX model in Settings → Engine & Model." + case .modelNotDownloaded(let id): return "Download \(id.value) in Settings → Engine & Model." + case .modelIncompatible(let id, let reason): return "\(id.value) can't run on MLX: \(reason)" + } + } +} + +/// The MLX server's lifecycle as the router, Settings and the menu bar see it. +nonisolated enum MLXServerState: Equatable, Sendable { + case idle + case unavailable(MLXUnavailableReason) + case starting(modelID: String) + case ready(port: Int, modelID: String) + case failed(reason: String, logTail: [String]) + case stopping + + var summary: String { + switch self { + case .idle: return "Not running" + case .unavailable(let reason): return reason.message + case .starting(let modelID): return "Loading \(modelID)…" + case .ready(_, let modelID): return "Running \(modelID)" + case .failed(let reason, _): return reason + case .stopping: return "Stopping…" + } + } + + /// Text for the Settings attention dot and callout. Nil while the server is running, loading, + /// stopping, or simply not selected: none of those need the user to act. + var problem: String? { + switch self { + case .unavailable(let reason): return reason.message + case .failed(let reason, _): return reason + case .idle, .starting, .ready, .stopping: return nil + } + } + + /// The `.unavailable` message a suggestion request gets while the server is not ready. + var notReadyMessage: String { + switch self { + case .starting(let modelID): return "MLX is loading \(modelID)…" + case .unavailable(let reason): return reason.message + case .failed(let reason, _): return "MLX stopped: \(reason)" + case .idle, .stopping: return "MLX is not running." + case .ready: return "MLX is ready." + } + } +} + +/// One model download as the pane shows it. +nonisolated struct MLXDownloadState: Equatable, Sendable { + let repoID: MLXRepoID + /// 0...1 once the total size is known; nil before that. + var fraction: Double? + var failure: String? + var isActive: Bool +} + +/// One running `mlx_lm.server` as `MLXServerManager` sees it. A protocol so tests drive the +/// manager with a fake process instead of Python. +protocol MLXRunningProcess: AnyObject { + var processIdentifier: Int32 { get } + var isRunning: Bool { get } + /// SIGTERM: ask the server to exit. + func terminate() + /// SIGKILL: used only after the grace period. + func forceKill() + /// The last lines the server wrote to stderr, for the failure message. + func stderrTail() -> [String] +} + +/// Starts the server process. `onExit` is delivered on the main actor with the exit status. +protocol MLXProcessLaunching { + func launch( + executable: URL, + arguments: [String], + environment: [String: String], + onExit: @escaping @MainActor (Int32) -> Void + ) throws -> any MLXRunningProcess +} + +/// Waits until a launched server answers `/v1/models` and has finished a warm-up completion. +protocol MLXReadinessProbing { + func waitUntilReady(port: Int, modelID: String, timeout: Duration) async throws +} diff --git a/Cotabby/Models/Runtime/Metrics/MenuResourceMonitor.swift b/Cotabby/Models/Runtime/Metrics/MenuResourceMonitor.swift new file mode 100644 index 000000000..c95817d78 --- /dev/null +++ b/Cotabby/Models/Runtime/Metrics/MenuResourceMonitor.swift @@ -0,0 +1,88 @@ +import Combine +import Foundation + +/// File overview: +/// Publishes `ResourceUsageSummary` for the menu bar popover. Unlike `SystemMetricsStore` it keeps +/// no history: it holds only the latest value and polls only between `start()` and `stop()`, which +/// the popover calls when its window is presented or dismissed (and from `.onAppear` / +/// `.onDisappear` as a fallback), so a closed menu costs nothing. One instance +/// lives for the app's lifetime (built by `CotabbyAppEnvironment`). +@MainActor +final class MenuResourceMonitor: ObservableObject { + nonisolated static let defaultInterval: TimeInterval = 2.0 + + @Published private(set) var summary: ResourceUsageSummary? + + private let interval: TimeInterval + private let sample: () -> ResourceUsageSummary + private let onStop: () -> Void + private let primingDelay: TimeInterval + private var timer: Timer? + private var isActive = false + + init( + interval: TimeInterval = MenuResourceMonitor.defaultInterval, + sample: @escaping () -> ResourceUsageSummary, + primingDelay: TimeInterval = 0.3, + onStop: @escaping () -> Void = {} + ) { + self.primingDelay = primingDelay + self.interval = interval + self.sample = sample + self.onStop = onStop + } + + nonisolated deinit { + // Same reasoning as `SystemMetricsStore`: no cleanup from a nonisolated deinit; the timer + // holds `self` weakly and invalidates itself on its first tick after we are gone. + } + + /// Seeds the CPU/GPU baselines with an unpublished reading, then publishes the first real one + /// after `primingDelay` and every `interval` after that. Publishing the priming reading itself + /// would show RAM (and "incl. MLX") without CPU or GPU deltas, which need two readings. + /// Idempotent: a second call while active does nothing. + func start() { + guard !isActive else { return } + isActive = true + _ = sample() + let first = Timer(timeInterval: primingDelay, repeats: false) { [weak self] _ in + guard let self else { return } + MainActor.assumeIsolated { + guard self.isActive else { return } + self.tick() + self.startRepeating() + } + } + RunLoop.main.add(first, forMode: .common) + timer = first + } + + private func startRepeating() { + let timer = Timer(timeInterval: interval, repeats: true) { [weak self] fired in + guard let self else { + fired.invalidate() + return + } + MainActor.assumeIsolated { self.tick() } + } + RunLoop.main.add(timer, forMode: .common) + self.timer = timer + } + + /// Idempotent: only the first call after `start()` clears state and notifies. + func stop() { + guard isActive else { return } + isActive = false + timer?.invalidate() + timer = nil + summary = nil + onStop() + } + + /// One reading. Internal so tests can drive it without waiting for a real timer. + func tick() { + summary = sample() + } + + var isRunning: Bool { isActive } +} diff --git a/Cotabby/Models/Runtime/Metrics/SystemMetricsStore.swift b/Cotabby/Models/Runtime/Metrics/SystemMetricsStore.swift index f0fff09d1..92bdceec1 100644 --- a/Cotabby/Models/Runtime/Metrics/SystemMetricsStore.swift +++ b/Cotabby/Models/Runtime/Metrics/SystemMetricsStore.swift @@ -24,6 +24,8 @@ struct SystemMetricSample: Identifiable, Equatable { let deviceGPUPercent: Double? /// GPU memory in use across the whole Mac, in bytes. let gpuMemoryBytes: UInt64? + /// True when `cpuPercent` and `footprintBytes` also count the MLX server process. + var includesMLXServer = false } @MainActor @@ -150,7 +152,8 @@ final class SystemMetricsStore: ObservableObject { footprintBytes: reading.footprintBytes, gpuPercent: gpuPercent, deviceGPUPercent: gpu.deviceUtilizationPercent, - gpuMemoryBytes: gpu.inUseMemoryBytes + gpuMemoryBytes: gpu.inUseMemoryBytes, + includesMLXServer: reading.includesMLXServer ) nextSampleID &+= 1 var updated = samples diff --git a/Cotabby/Models/Settings/SuggestionSettingsData.swift b/Cotabby/Models/Settings/SuggestionSettingsData.swift index ef8882ef6..2c5700a55 100644 --- a/Cotabby/Models/Settings/SuggestionSettingsData.swift +++ b/Cotabby/Models/Settings/SuggestionSettingsData.swift @@ -33,6 +33,12 @@ struct SuggestionEngineSettings: Equatable { /// Keep that fallback model loaded while Apple Intelligence is selected, so the first /// fallback suggestion does not wait for the model to load. Costs the model's memory. var keepsFallbackModelLoaded: Bool + /// Hugging Face repo id of the MLX model, exactly as `mlx_lm.server --model` receives it. + var mlxModelID: String = "" + /// Upper bound on the MLX server's prompt (KV) cache, in bytes. + var mlxPromptCacheBytes: Int = SuggestionSettingsStore.defaultMLXPromptCacheBytes + var batteryMLXModelID: String = "" + var pluggedInMLXModelID: String = "" } /// Completion length, timing, streaming, and acceptance behavior. @@ -190,6 +196,26 @@ extension SuggestionSettingsData { set { engine.openAICompatibleAPIMode = newValue } } + var mlxModelID: String { + get { engine.mlxModelID } + set { engine.mlxModelID = newValue } + } + + var mlxPromptCacheBytes: Int { + get { engine.mlxPromptCacheBytes } + set { engine.mlxPromptCacheBytes = newValue } + } + + var batteryMLXModelID: String { + get { engine.batteryMLXModelID } + set { engine.batteryMLXModelID = newValue } + } + + var pluggedInMLXModelID: String { + get { engine.pluggedInMLXModelID } + set { engine.pluggedInMLXModelID = newValue } + } + var isPowerBasedModelSwitchingEnabled: Bool { get { engine.isPowerBasedModelSwitchingEnabled } set { engine.isPowerBasedModelSwitchingEnabled = newValue } diff --git a/Cotabby/Models/Settings/SuggestionSettingsModel.swift b/Cotabby/Models/Settings/SuggestionSettingsModel.swift index ca481a454..f99211307 100644 --- a/Cotabby/Models/Settings/SuggestionSettingsModel.swift +++ b/Cotabby/Models/Settings/SuggestionSettingsModel.swift @@ -63,6 +63,10 @@ final class SuggestionSettingsModel: ObservableObject { @Published private(set) var openAICompatibleBaseURL: String @Published private(set) var openAICompatibleModelName: String @Published private(set) var openAICompatibleAPIMode: OpenAICompatibleAPIMode + /// MLX model repo id and prompt-cache cap. Read by the MLX server wiring in + /// `CotabbyAppEnvironment`; changing either restarts the server. + @Published private(set) var mlxModelID: String + @Published private(set) var mlxPromptCacheBytes: Int /// Non-secret change token that lets lifecycle observers react to Keychain updates without /// publishing the credential itself. @Published private(set) var endpointCredentialRevision: UInt64 = 0 @@ -173,6 +177,8 @@ final class SuggestionSettingsModel: ObservableObject { @Published private(set) var pluggedInEngine: SuggestionEngineKind @Published private(set) var pluggedInModelFilename: String @Published private(set) var pluggedInEndpointModelName: String + @Published private(set) var batteryMLXModelID: String + @Published private(set) var pluggedInMLXModelID: String /// Owns the on-disk keys, defaults, migrations, and per-field writes. The facade holds one and /// routes every load and save through it. @@ -304,6 +310,10 @@ final class SuggestionSettingsModel: ObservableObject { pluggedInEngine = data.pluggedInEngine pluggedInModelFilename = data.pluggedInModelFilename pluggedInEndpointModelName = data.pluggedInEndpointModelName + mlxModelID = data.mlxModelID + mlxPromptCacheBytes = data.mlxPromptCacheBytes + batteryMLXModelID = data.batteryMLXModelID + pluggedInMLXModelID = data.pluggedInMLXModelID schedulePauseExpirationIfNeeded() } @@ -390,6 +400,10 @@ final class SuggestionSettingsModel: ObservableObject { pluggedInEngine = data.pluggedInEngine pluggedInModelFilename = data.pluggedInModelFilename pluggedInEndpointModelName = data.pluggedInEndpointModelName + mlxModelID = data.mlxModelID + mlxPromptCacheBytes = data.mlxPromptCacheBytes + batteryMLXModelID = data.batteryMLXModelID + pluggedInMLXModelID = data.pluggedInMLXModelID do { try endpointCredentialStore.deleteAPIKey() endpointCredentialRevision &+= 1 @@ -427,7 +441,11 @@ final class SuggestionSettingsModel: ObservableObject { pluggedInModelFilename: pluggedInModelFilename, pluggedInEndpointModelName: pluggedInEndpointModelName, isAppleLanguageFallbackEnabled: isAppleLanguageFallbackEnabled, - keepsFallbackModelLoaded: keepsFallbackModelLoaded + keepsFallbackModelLoaded: keepsFallbackModelLoaded, + mlxModelID: mlxModelID, + mlxPromptCacheBytes: mlxPromptCacheBytes, + batteryMLXModelID: batteryMLXModelID, + pluggedInMLXModelID: pluggedInMLXModelID ), completion: SuggestionCompletionSettings( selectedWordCountPreset: selectedWordCountPreset, @@ -574,6 +592,32 @@ final class SuggestionSettingsModel: ObservableObject { store.saveOpenAICompatibleAPIMode(mode) } + func setMLXModelID(_ modelID: String) { + let trimmed = modelID.trimmingCharacters(in: .whitespacesAndNewlines) + guard mlxModelID != trimmed else { return } + mlxModelID = trimmed + store.saveMLXModelID(trimmed) + } + + func setMLXPromptCacheBytes(_ bytes: Int) { + let clamped = SuggestionSettingsStore.clampedMLXPromptCacheBytes(bytes) + guard mlxPromptCacheBytes != clamped else { return } + mlxPromptCacheBytes = clamped + store.saveMLXPromptCacheBytes(clamped) + } + + func setBatteryMLXModelID(_ modelID: String) { + guard batteryMLXModelID != modelID else { return } + batteryMLXModelID = modelID + store.saveBatteryMLXModelID(modelID) + } + + func setPluggedInMLXModelID(_ modelID: String) { + guard pluggedInMLXModelID != modelID else { return } + pluggedInMLXModelID = modelID + store.savePluggedInMLXModelID(modelID) + } + var openAICompatibleConfiguration: OpenAICompatibleEndpointConfiguration { get throws { try OpenAICompatibleEndpointConfiguration( @@ -700,6 +744,7 @@ final class SuggestionSettingsModel: ObservableObject { case .appleIntelligence: return .appleIntelligence case .llamaOpenSource: return .llama(filename: batteryModelFilename) case .openAICompatible: return .openAICompatible(modelName: batteryEndpointModelName) + case .mlx: return .mlx(modelID: batteryMLXModelID) } } @@ -709,6 +754,7 @@ final class SuggestionSettingsModel: ObservableObject { case .appleIntelligence: return .appleIntelligence case .llamaOpenSource: return .llama(filename: pluggedInModelFilename) case .openAICompatible: return .openAICompatible(modelName: pluggedInEndpointModelName) + case .mlx: return .mlx(modelID: pluggedInMLXModelID) } } @@ -720,6 +766,9 @@ final class SuggestionSettingsModel: ObservableObject { if case .openAICompatible(let modelName) = profile { setBatteryEndpointModelName(modelName) } + if case .mlx(let modelID) = profile { + setBatteryMLXModelID(modelID) + } } func setPluggedInProfile(_ profile: PowerProfile) { @@ -730,6 +779,9 @@ final class SuggestionSettingsModel: ObservableObject { if case .openAICompatible(let modelName) = profile { setPluggedInEndpointModelName(modelName) } + if case .mlx(let modelID) = profile { + setPluggedInMLXModelID(modelID) + } } /// Seeds each per-power-source profile from the active engine + model the first time the feature @@ -739,7 +791,8 @@ final class SuggestionSettingsModel: ObservableObject { func initializePowerProfiles( currentEngine: SuggestionEngineKind, currentModelFilename: String?, - currentEndpointModelName: String? = nil + currentEndpointModelName: String? = nil, + currentMLXModelID: String? = nil ) { if batteryEngine == .llamaOpenSource, batteryModelFilename.isEmpty { setBatteryEngine(currentEngine) @@ -747,6 +800,8 @@ final class SuggestionSettingsModel: ObservableObject { setBatteryModelFilename(currentModelFilename) } else if currentEngine == .openAICompatible, let currentEndpointModelName { setBatteryEndpointModelName(currentEndpointModelName) + } else if currentEngine == .mlx, let currentMLXModelID { + setBatteryMLXModelID(currentMLXModelID) } } @@ -756,6 +811,8 @@ final class SuggestionSettingsModel: ObservableObject { setPluggedInModelFilename(currentModelFilename) } else if currentEngine == .openAICompatible, let currentEndpointModelName { setPluggedInEndpointModelName(currentEndpointModelName) + } else if currentEngine == .mlx, let currentMLXModelID { + setPluggedInMLXModelID(currentMLXModelID) } } } diff --git a/Cotabby/Models/Suggestion/SuggestionEngineModels.swift b/Cotabby/Models/Suggestion/SuggestionEngineModels.swift index 1478b6217..82f2e5818 100644 --- a/Cotabby/Models/Suggestion/SuggestionEngineModels.swift +++ b/Cotabby/Models/Suggestion/SuggestionEngineModels.swift @@ -11,6 +11,7 @@ enum SuggestionEngineKind: String, CaseIterable, Equatable, Hashable, Sendable, case appleIntelligence case llamaOpenSource case openAICompatible + case mlx var id: String { rawValue } @@ -22,6 +23,8 @@ enum SuggestionEngineKind: String, CaseIterable, Equatable, Hashable, Sendable, return "Open Source" case .openAICompatible: return "Local Endpoint" + case .mlx: + return "MLX" } } @@ -36,9 +39,12 @@ enum SuggestionEngineKind: String, CaseIterable, Equatable, Hashable, Sendable, return "cpu.fill" case .openAICompatible: return "network" + case .mlx: + return "bolt.fill" } } + /// MLX has its own Settings section; the GGUF model list does not apply to it. var supportsLocalModelManagement: Bool { switch self { case .appleIntelligence: @@ -47,19 +53,49 @@ enum SuggestionEngineKind: String, CaseIterable, Equatable, Hashable, Sendable, return true case .openAICompatible: return false + case .mlx: + return false } } + /// Whether a request to this engine stays on this Mac. Gates private context: typing history, + /// the document tail, and the on-device visual-context limits. MLX speaks HTTP, but only to a + /// 127.0.0.1 server Cotabby launched itself, so it is on-device. + var isOnDevice: Bool { + switch self { + case .appleIntelligence, .llamaOpenSource, .mlx: + return true + case .openAICompatible: + return false + } + } + + /// Whether Cotabby may start generations the user has not asked for yet: predict-ahead while + /// typing and speculative continuation after an accept. Both rely on the in-process runtimes, + /// where a speculative decode yields to the real request at once. A server is shared work: + /// mlx_lm.server runs every request on one generation thread (queued or batched beside the + /// request the user is waiting for) and drops a cancelled one only after its handler's next + /// write fails, so speculation there delays real suggestions. The endpoint would also receive + /// text the user never typed. + var allowsSpeculativeGeneration: Bool { + switch self { + case .appleIntelligence, .llamaOpenSource: + return true + case .openAICompatible, .mlx: + return false + } + } } /// A per-power-source suggestion configuration: which engine to use and, for the local llama engine, -/// which downloaded model file. Apple Intelligence carries no model file because the OS owns the -/// model. Used by the power-based switching feature to pick an engine + model per power state, and as -/// the single selection tag for the per-state pickers in Settings. +/// which downloaded model file, and for MLX, which Hugging Face repo id. Apple Intelligence carries no +/// model file because the OS owns the model. Used by the power-based switching feature to pick an +/// engine + model per power state, and as the single selection tag for the per-state pickers in Settings. enum PowerProfile: Equatable, Hashable { case appleIntelligence case llama(filename: String) case openAICompatible(modelName: String) + case mlx(modelID: String) /// The engine this profile selects. Settings persists engine + filename separately, so this is /// the bridge from those two stored fields to a single picker selection (and back). @@ -71,6 +107,8 @@ enum PowerProfile: Equatable, Hashable { return .llamaOpenSource case .openAICompatible: return .openAICompatible + case .mlx: + return .mlx } } } diff --git a/Cotabby/Services/History/TypingHistoryStore.swift b/Cotabby/Services/History/TypingHistoryStore.swift index e16a5e11a..af46b0e7c 100644 --- a/Cotabby/Services/History/TypingHistoryStore.swift +++ b/Cotabby/Services/History/TypingHistoryStore.swift @@ -533,7 +533,7 @@ final class TypingHistoryStore: ObservableObject, SuggestionHistoryProviding { // MARK: - SuggestionHistoryProviding func historyExamples(for context: FocusedInputContext, engine: SuggestionEngineKind) -> [String] { - guard preferences.isUsingHistory, engine != .openAICompatible, let index else { return [] } + guard preferences.isUsingHistory, engine.isOnDevice, let index else { return [] } let stableText = TypingHistoryQuery.stableText(from: context.precedingText) // The window title (an email subject, a document name) is the most stable topical signal a // field has, so it joins the query even before the first full block of words is typed. @@ -557,7 +557,7 @@ final class TypingHistoryStore: ObservableObject, SuggestionHistoryProviding { } func phraseContinuation(for request: SuggestionRequest, engine: SuggestionEngineKind) -> String? { - guard preferences.isUsingHistory, engine != .openAICompatible, let phrases else { return nil } + guard preferences.isUsingHistory, engine.isOnDevice, let phrases else { return nil } // Text after the caret on the same line would be pushed along by an inserted phrase; leave // those mid-line positions to the model, which sees the trailing text. let restOfLine = request.context.trailingText.prefix { !$0.isNewline } diff --git a/Cotabby/Services/ModelManagement/HuggingFaceAPIClient.swift b/Cotabby/Services/ModelManagement/HuggingFaceAPIClient.swift index c1d325495..092960075 100644 --- a/Cotabby/Services/ModelManagement/HuggingFaceAPIClient.swift +++ b/Cotabby/Services/ModelManagement/HuggingFaceAPIClient.swift @@ -68,6 +68,33 @@ enum HuggingFaceAPIClient { } } + /// Whether downloading the repo needs an accepted license and a token. The model-info API + /// reports `gated` as `false`, `"auto"` or `"manual"`. + static func isGated(repoId: String) async throws -> Bool { + guard var components = URLComponents(string: "https://huggingface.co") else { + throw APIError.invalidURL + } + components.path = "/api/models/\(repoId)" + guard let url = components.url else { throw APIError.invalidURL } + let data = try await performRequest(url: url) + guard let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { + throw APIError.decodingFailed(URLError(.cannotParseResponse)) + } + if let gated = object["gated"] as? Bool { return gated } + return object["gated"] is String + } + + /// One file of a repo at `main` (`resolve` redirects LFS files to the CDN; URLSession follows). + /// Used for small files such as `config.json`; a missing file surfaces as `httpError(404)`. + static func fetchFileData(repoId: String, path: String) async throws -> Data { + guard var components = URLComponents(string: "https://huggingface.co") else { + throw APIError.invalidURL + } + components.path = "/\(repoId)/resolve/main/\(path)" + guard let url = components.url else { throw APIError.invalidURL } + return try await performRequest(url: url) + } + private static func performRequest(url: URL) async throws -> Data { let configuration = URLSessionConfiguration.ephemeral configuration.requestCachePolicy = .reloadIgnoringLocalCacheData diff --git a/Cotabby/Services/Runtime/MLX/MLXHealthProbe.swift b/Cotabby/Services/Runtime/MLX/MLXHealthProbe.swift new file mode 100644 index 000000000..2522c9fed --- /dev/null +++ b/Cotabby/Services/Runtime/MLX/MLXHealthProbe.swift @@ -0,0 +1,43 @@ +import Foundation + +/// File overview: +/// Asks a running `mlx_lm.server` whether its generator is alive. A 404 on a completion does not +/// prove it died: mlx_lm 0.32.0 turns any exception inside `generate()` (a template error on one +/// bad prompt, say) into a 404 too. `GET /health` is the exact signal: it answers 503 only when +/// `generation_available()` is false and 200 otherwise. The MLX engine calls this before asking +/// the manager to restart, so one bad request cannot restart a healthy server. + +/// What `/health` said. +nonisolated enum MLXServerHealth: Equatable, Sendable { + case healthy + case generatorDead + /// Refused, timed out or any other transport failure: the server is not answering at all. + case unreachable + /// The probe itself was cancelled (the caller moved on). It says nothing about the server. + case cancelled +} + +/// Injected into the engine so tests answer without a socket. +typealias MLXHealthCheck = @Sendable (Int) async -> MLXServerHealth + +nonisolated enum MLXHealthProbe { + static func check(port: Int) async -> MLXServerHealth { + guard let url = URL(string: "http://127.0.0.1:\(port)/health") else { return .unreachable } + var request = URLRequest(url: url) + request.timeoutInterval = 2 + do { + let (_, response) = try await URLSession.shared.data(for: request) + switch (response as? HTTPURLResponse)?.statusCode { + case 503: return .generatorDead + case 200: return .healthy + default: return .unreachable + } + } catch is CancellationError { + return .cancelled + } catch let error as URLError where error.code == .cancelled { + return .cancelled + } catch { + return Task.isCancelled ? .cancelled : .unreachable + } + } +} diff --git a/Cotabby/Services/Runtime/MLX/MLXModelDownloader.swift b/Cotabby/Services/Runtime/MLX/MLXModelDownloader.swift new file mode 100644 index 000000000..9bdb2fb83 --- /dev/null +++ b/Cotabby/Services/Runtime/MLX/MLXModelDownloader.swift @@ -0,0 +1,292 @@ +import Combine +import Foundation +import Logging + +/// File overview: +/// Owns everything about MLX model files: +/// - which usable models the Hugging Face cache holds (`cachedModels`); +/// - one download at a time, with byte progress and cancel; +/// - checking a custom repo id before downloading it; +/// - deleting a model. +/// Downloads run the toolchain's own Python with `huggingface_hub.snapshot_download`, so files +/// land exactly where `mlx_lm.server` looks and a cancelled download resumes next time. Owned by +/// `CotabbyAppEnvironment`; the Settings pane observes it. + +/// The answer to "can Cotabby download this custom repo?", shown under the Settings text field. +/// A value type because the pane only renders it; it owns no state and no network access. +nonisolated enum MLXRepoValidation: Equatable, Sendable { + case valid(MLXRepoID, totalBytes: Int64, isInstruct: Bool) + case invalidID + case notFoundOrPrivate(MLXRepoID) + case gated(MLXRepoID) + case incompatible(MLXRepoID, MLXModelCompatibility.Reason) + case networkError(String) + + /// Nil for `.valid`; otherwise the sentence the pane shows under the field. + var message: String? { + switch self { + case .valid: return nil + case .invalidID: return "Enter a Hugging Face model id like mlx-community/gemma-4-e4b-5bit." + case .notFoundOrPrivate(let id): return "\(id.value) was not found, or it is private." + case .gated(let id): return "\(id.value) needs a Hugging Face login, which Cotabby does not support yet." + case .incompatible(let id, let reason): return "\(id.value): \(reason.message)" + case .networkError(let message): return "Could not reach Hugging Face: \(message)" + } + } +} + +/// The download and cache manager behind the MLX model list. It is `@MainActor` because the +/// Settings pane observes its published state; the heavy work (directory scans, the Python +/// download itself) runs off the main actor and only the resulting values are published back. +/// Its collaborators are injected closures, so tests replace the network and the process with fakes. +@MainActor +final class MLXModelDownloader: ObservableObject { + typealias RepoFilesFetch = (String) async throws -> [HFRepoFile] + typealias GatedFetch = (String) async throws -> Bool + /// Downloads one small file (`config.json`) of a remote repo: repo id, then file path. + typealias ConfigFetch = (String) async throws -> Data + typealias RunProcess = (URL, [String], [String: String], Duration) async throws -> MLXProcessResult + /// The cache scan, injectable so a test can hold one scan after it has read the disk. + typealias CacheScan = @Sendable (URL, Set?) async -> MLXCacheScanResult + + /// Nil until the first scan finishes, so the server wiring waits instead of reporting the + /// selected model as missing during launch. + @Published private(set) var cachedModels: [MLXCachedModel]? + /// Complete downloads the compatibility rule turned away, with the reason. Published together + /// with `cachedModels` so the UI and the server wiring never see one without the other. + @Published private(set) var rejectedModels: [MLXRepoID: MLXModelCompatibility.Reason] = [:] + @Published private(set) var download: MLXDownloadState? + + private let cacheRoot: URL + private let fileManager: FileManager + private let fetchRepoFiles: RepoFilesFetch + private let fetchIsGated: GatedFetch + private let fetchConfig: ConfigFetch + private let runProcess: RunProcess + private let pollInterval: Duration + private let scanCache: CacheScan + /// The architectures the installed mlx-lm supports, cached per Python (asked once, ~1 s). + private var supportedTypes: (python: URL, types: Set?)? + private var downloadTask: Task? + /// Bumped by every `refreshCache`. Scans run off the main actor, so two can overlap; a scan + /// that started before a delete could finish last and publish the deleted model again. Only + /// the newest refresh may publish. + private var refreshGeneration: UInt64 = 0 + + init( + cacheRoot: URL, + fileManager: FileManager = .default, + fetchRepoFiles: @escaping RepoFilesFetch = { try await HuggingFaceAPIClient.fetchRepoFiles(repoId: $0) }, + fetchIsGated: @escaping GatedFetch = { try await HuggingFaceAPIClient.isGated(repoId: $0) }, + fetchConfig: @escaping ConfigFetch = { + try await HuggingFaceAPIClient.fetchFileData(repoId: $0, path: "config.json") + }, + runProcess: @escaping RunProcess = { executable, arguments, environment, timeout in + try await MLXProcessRunner.run( + executable: executable, arguments: arguments, environment: environment, timeout: timeout) + }, + pollInterval: Duration = .milliseconds(500), + scanCache: @escaping CacheScan = { root, types in + MLXCacheScanner.scanDetailed(cacheRoot: root, supportedModelTypes: types) + } + ) { + self.cacheRoot = cacheRoot + self.fileManager = fileManager + self.fetchRepoFiles = fetchRepoFiles + self.fetchIsGated = fetchIsGated + self.fetchConfig = fetchConfig + self.runProcess = runProcess + self.pollInterval = pollInterval + self.scanCache = scanCache + } + + static func live() -> MLXModelDownloader { + MLXModelDownloader(cacheRoot: MLXCacheScanner.cacheRoot( + environment: ProcessInfo.processInfo.environment, + home: FileManager.default.homeDirectoryForCurrentUser + )) + } + + func refreshCache(toolchain: MLXToolchain?) async { + refreshGeneration &+= 1 + let generation = refreshGeneration + let types = await supportedModelTypes(for: toolchain) + let root = cacheRoot + let scan = scanCache + let result = await Task.detached(priority: .utility) { + await scan(root, types) + }.value + // A newer refresh started while this one was suspended, so its answer is the current one. + // Publishing this older scan could bring back a model that was just deleted. + guard generation == refreshGeneration else { return } + cachedModels = result.models + rejectedModels = result.rejected + } + + /// Checks a custom repo before any download, with the same `MLXModelCompatibility` rule the + /// cache scan uses, so "accepted here" and "listed after download" cannot disagree. The repo's + /// `config.json` (a few KB) is fetched for the architecture and head checks; a 404 there means + /// the repo has no config at all. + func validateCustomRepo(_ raw: String, toolchain: MLXToolchain?) async -> MLXRepoValidation { + guard let repoID = MLXRepoID(raw) else { return .invalidID } + do { + let files = try await fetchRepoFiles(repoID.value) + if try await fetchIsGated(repoID.value) { return .gated(repoID) } + var config: [String: Any]? + if files.contains(where: { $0.path == "config.json" }) { + do { + let data = try await fetchConfig(repoID.value) + config = (try? JSONSerialization.jsonObject(with: data)) as? [String: Any] + } catch HuggingFaceAPIClient.APIError.httpError(404) { + return .incompatible(repoID, .missingConfig) + } + } + let types = await supportedModelTypes(for: toolchain) + switch MLXModelCompatibility.verdict( + repoName: repoID.name, fileNames: files.map(\.path), config: config, supportedModelTypes: types + ) { + case .incompatible(let reason): + return .incompatible(repoID, reason) + case .compatible(_, let isInstruct): + return .valid(repoID, totalBytes: files.reduce(Int64(0)) { $0 + $1.size }, isInstruct: isInstruct) + } + } catch HuggingFaceAPIClient.APIError.httpError(let code) where [401, 403, 404].contains(code) { + return .notFoundOrPrivate(repoID) + } catch { + return .networkError(error.localizedDescription) + } + } + + /// Starts one download; ignored while another runs. + func startDownload(_ repoID: MLXRepoID, toolchain: MLXToolchain) { + guard downloadTask == nil else { return } + download = MLXDownloadState(repoID: repoID, fraction: nil, failure: nil, isActive: true) + CotabbyLogger.runtime.info("MLX model download started", metadata: ["model": .string(repoID.value)]) + downloadTask = Task { [weak self] in + await self?.runDownload(repoID, toolchain: toolchain) + } + } + + /// Terminates the download process. huggingface_hub resumes the partial files next time. + func cancelDownload() { + downloadTask?.cancel() + } + + /// Removes the repo from the shared cache, then rescans (the server wiring reacts if it was + /// the selected model). + func delete(_ repoID: MLXRepoID, toolchain: MLXToolchain?) async { + // A download still writing into this folder would bring the model back, so stop it and wait + // until it has really finished before removing anything. + if download?.repoID == repoID, download?.isActive == true { + let task = downloadTask + task?.cancel() + await task?.value + } + let folder = cacheRoot.appendingPathComponent(repoID.cacheFolderName, isDirectory: true) + let manager = fileManager + // Model folders are several GB; removing them on the main actor would stall the UI. + let failure: String? = await Task.detached(priority: .utility) { () -> String? in + guard manager.fileExists(atPath: folder.path) else { return nil } + do { + try manager.removeItem(at: folder) + return nil + } catch { + return error.localizedDescription + } + }.value + if let failure { + CotabbyLogger.runtime.error( + "Could not delete MLX model", + metadata: ["model": .string(repoID.value), "error": .string(failure)]) + } else { + CotabbyLogger.runtime.info("Deleted MLX model", metadata: ["model": .string(repoID.value)]) + } + await refreshCache(toolchain: toolchain) + } + + // MARK: - Private + + private func runDownload(_ repoID: MLXRepoID, toolchain: MLXToolchain) async { + let total = (try? await fetchRepoFiles(repoID.value))?.reduce(Int64(0)) { $0 + $1.size } ?? 0 + let root = cacheRoot + let poller = Task { [weak self] in + while !Task.isCancelled { + let bytes = await Task.detached(priority: .utility) { + MLXCacheScanner.downloadedBytes(repo: repoID, cacheRoot: root) + }.value + guard let self, self.download?.repoID == repoID, self.download?.isActive == true else { return } + self.download?.fraction = MLXCacheScanner.progress(downloadedBytes: bytes, totalBytes: total) + try? await Task.sleep(for: self.pollInterval) + } + } + + let outcome: Result + do { + outcome = .success(try await runProcess( + toolchain.pythonExecutable, + ["-I", "-c", MLXToolchainRules.downloadScript, repoID.value], + MLXToolchainRules.downloadEnvironment(base: ProcessInfo.processInfo.environment), + .seconds(6 * 60 * 60) + )) + } catch { + outcome = .failure(error) + } + poller.cancel() + + switch outcome { + case .success(let result) where result.status == 0: + // Cleared after the rescan below, once we know the model is actually usable. + CotabbyLogger.runtime.info("MLX model downloaded", metadata: ["model": .string(repoID.value)]) + case .success(let result): + download = MLXDownloadState( + repoID: repoID, fraction: download?.fraction, + failure: result.stderrTail.last ?? "The download failed.", isActive: false) + case .failure(let error) where error is CancellationError: + download = nil + case .failure(let error): + download = MLXDownloadState( + repoID: repoID, fraction: download?.fraction, failure: error.localizedDescription, isActive: false) + } + downloadTask = nil + await refreshCache(toolchain: toolchain) + // A finished download that the rescan does not list is one the installed mlx-lm cannot run + // (its architecture is unsupported); clearing the card silently would look like a no-op. + if case .success(let result) = outcome, result.status == 0, download?.repoID == repoID { + if cachedModels?.contains(where: { $0.repoID == repoID }) == true { + download = nil + } else { + // No rejection means the snapshot is missing files, i.e. the download is incomplete. + let failure = rejectedModels[repoID].map { "Downloaded, but it can't be used. \($0.message)" } + ?? "The download did not complete. Try again." + download = MLXDownloadState( + repoID: repoID, fraction: 1, + failure: failure, + isActive: false) + } + } + } + + /// The architectures the installed mlx-lm supports, nil without a toolchain or when the probe + /// failed. Cached per Python so the ~1 s probe runs once. + private func supportedModelTypes(for toolchain: MLXToolchain?) async -> Set? { + guard let toolchain else { return nil } + if let cached = supportedTypes, cached.python == toolchain.pythonExecutable { return cached.types } + let types = await loadSupportedTypes(python: toolchain.pythonExecutable) + // Cache only a real answer: a probe that failed or was cancelled must be retried + // next time, not remembered as "unknown" (which turns the type filter off). + if types != nil { supportedTypes = (toolchain.pythonExecutable, types) } + return types + } + + private func loadSupportedTypes(python: URL) async -> Set? { + guard let result = try? await runProcess( + python, + ["-I", "-c", MLXToolchainRules.supportedModelTypesScript], + MLXToolchainRules.sanitizedEnvironment(ProcessInfo.processInfo.environment), + .seconds(30) + ), result.status == 0, + let names = try? JSONDecoder().decode([String].self, from: Data(result.stdout.utf8)) + else { return nil } + return Set(names) + } +} diff --git a/Cotabby/Services/Runtime/MLX/MLXProcessRunner.swift b/Cotabby/Services/Runtime/MLX/MLXProcessRunner.swift new file mode 100644 index 000000000..e4ffd4151 --- /dev/null +++ b/Cotabby/Services/Runtime/MLX/MLXProcessRunner.swift @@ -0,0 +1,253 @@ +import Foundation + +/// File overview: +/// Runs one MLX helper process (uv, tar, a Python one-liner) to completion off the main actor, +/// with a timeout, cooperative cancellation and bounded output capture. Owned by nobody: callers +/// await `run` and get a value back. Kept separate from the long-lived server process +/// (`MLXServerManager`), which never "completes". + +nonisolated struct MLXProcessResult: Equatable, Sendable { + let status: Int32 + let stdout: String + let stderrTail: [String] +} + +nonisolated enum MLXProcessRunnerError: LocalizedError, Equatable { + case timedOut(tail: [String]) + case launchFailed(String) + + var errorDescription: String? { + switch self { + case .timedOut: return "The MLX helper took too long and was stopped." + case .launchFailed(let message): return "The MLX helper could not start: \(message)" + } + } +} + +/// Collects a child's output while it runs. Both pipes must be drained continuously: a child +/// that fills a 64 KB pipe buffer blocks forever. Stdout is kept whole up to 1 MiB (callers read +/// short answers from it); stderr keeps only its last lines for error messages. +nonisolated final class MLXOutputCollector: @unchecked Sendable { + static let stdoutLimit = 1 << 20 + + private let lock = NSLock() + private let tailLines: Int + private var stdout = Data() + private var stderrLines: [String] = [] + /// Bytes after the last line break. Kept as bytes, not text, so a UTF-8 character split across + /// two pipe reads is decoded whole once its line is complete. + private var partialStderrLine = Data() + /// Cap on an unterminated stderr fragment (keeps the last bytes). + private static let partialLineLimit = 8 * 1024 + private var stdoutClosed = false + private var stderrClosed = false + + init(tailLines: Int) { + self.tailLines = tailLines + } + + /// True once both pipes have reached EOF. The runner polls this during its bounded drain. + var bothStreamsClosed: Bool { + lock.withLock { stdoutClosed && stderrClosed } + } + + func appendStdout(_ data: Data) { + lock.withLock { + let room = Self.stdoutLimit - stdout.count + if room > 0 { stdout.append(data.prefix(room)) } + } + } + + func appendStderr(_ data: Data) { + guard !data.isEmpty else { return } + lock.withLock { + // `\r` separates lines too: progress redraws (tqdm) overwrite one line with carriage + // returns and would otherwise grow a single "line" without bound. + partialStderrLine.append(data) + var lineStart = partialStderrLine.startIndex + for index in partialStderrLine.indices where Self.isLineBreak(partialStderrLine[index]) { + let line = partialStderrLine[lineStart.. tailLines { stderrLines.removeFirst(stderrLines.count - tailLines) } + } + } + + private static func isLineBreak(_ byte: UInt8) -> Bool { + byte == UInt8(ascii: "\n") || byte == UInt8(ascii: "\r") + } + + /// The last `partialLineLimit` bytes of an unterminated fragment. A cut can land inside a + /// multi-byte character, so leading UTF-8 continuation bytes (`10xxxxxx`) are dropped with it. + private static func boundedFragment(_ fragment: Data) -> Data { + guard fragment.count > partialLineLimit else { return Data(fragment) } + var kept = fragment.suffix(partialLineLimit) + while let first = kept.first, first & 0xC0 == 0x80 { kept = kept.dropFirst() } + return Data(kept) + } + + /// Decodes one line or answer. A child's output is almost always UTF-8; anything else is + /// reported by size rather than shown as mojibake. + static func text(_ bytes: Data) -> String { + String(bytes: bytes, encoding: .utf8) ?? "<\(bytes.count) bytes of non-UTF-8 output>" + } + + func markStdoutClosed() { + lock.withLock { stdoutClosed = true } + } + + func markStderrClosed() { + lock.withLock { stderrClosed = true } + } + + var stdoutText: String { + lock.withLock { Self.text(stdout) } + } + + var stderrTail: [String] { + lock.withLock { + let all = partialStderrLine.isEmpty ? stderrLines : stderrLines + [Self.text(partialStderrLine)] + return Array(all.suffix(tailLines)) + } + } +} + +/// Bridges `Process.terminationHandler` (a background thread) to one awaiting task, in either +/// order: the process may exit before anyone awaits. +nonisolated private final class ExitSignal: @unchecked Sendable { + private let lock = NSLock() + private var status: Int32? + private var continuation: CheckedContinuation? + + func finish(_ value: Int32) { + let waiting: CheckedContinuation? = lock.withLock { + status = value + defer { continuation = nil } + return continuation + } + waiting?.resume(returning: value) + } + + func wait() async -> Int32 { + await withCheckedContinuation { continuation in + let ready: Int32? = lock.withLock { + if let status { return status } + self.continuation = continuation + return nil + } + if let ready { continuation.resume(returning: ready) } + } + } +} + +nonisolated enum MLXProcessRunner { + static func run( + executable: URL, + arguments: [String], + environment: [String: String], + timeout: Duration, + tailLines: Int = 20 + ) async throws -> MLXProcessResult { + try Task.checkCancellation() + let process = Process() + process.executableURL = executable + process.arguments = arguments + process.environment = environment + process.standardInput = FileHandle.nullDevice + let stdoutPipe = Pipe() + let stderrPipe = Pipe() + process.standardOutput = stdoutPipe + process.standardError = stderrPipe + + let output = MLXOutputCollector(tailLines: tailLines) + // At EOF the handler is called with empty data, and it keeps being called until it is removed. + // Removing it at EOF and recording the close is how the drain below knows the pipe is done. + stdoutPipe.fileHandleForReading.readabilityHandler = { handle in + let data = handle.availableData + if data.isEmpty { + handle.readabilityHandler = nil + output.markStdoutClosed() + } else { + output.appendStdout(data) + } + } + stderrPipe.fileHandleForReading.readabilityHandler = { handle in + let data = handle.availableData + if data.isEmpty { + handle.readabilityHandler = nil + output.markStderrClosed() + } else { + output.appendStderr(data) + } + } + let exit = ExitSignal() + process.terminationHandler = { exit.finish($0.terminationStatus) } + + do { + try process.run() + } catch { + stdoutPipe.fileHandleForReading.readabilityHandler = nil + stderrPipe.fileHandleForReading.readabilityHandler = nil + throw MLXProcessRunnerError.launchFailed(error.localizedDescription) + } + + // Either the exit status or nil for "the timeout fired first". The group always waits + // for the exit task, so every path below makes sure the child actually ends. + let status: Int32? + do { + status = try await withTaskCancellationHandler { + try await withThrowingTaskGroup(of: Int32?.self) { group in + group.addTask { await exit.wait() } + group.addTask { + try await Task.sleep(for: timeout) + return nil + } + // `next()` is `Int32??`: nil when the group is empty, `.some(nil)` on timeout. + let first = try await group.next().flatMap { $0 } + if first == nil { Self.stop(process) } + group.cancelAll() + return first + } + } onCancel: { + Self.stop(process) + } + } catch { + await Self.drain(output, stdoutPipe, stderrPipe) + throw error + } + + await Self.drain(output, stdoutPipe, stderrPipe) + + try Task.checkCancellation() + guard let status else { + throw MLXProcessRunnerError.timedOut(tail: output.stderrTail) + } + return MLXProcessResult(status: status, stdout: output.stdoutText, stderrTail: output.stderrTail) + } + + /// Waits a bounded grace (about 1 s) for both pipes to reach EOF, then detaches the handlers. + /// The direct child may be gone while a grandchild (a `sleep &`, a uv build backend) still holds + /// the write end open. A blocking read would then wait forever and could not be cancelled, so + /// the drain polls the collector instead and abandons whatever has not arrived after the grace. + /// A cancelled task skips the wait: its sleep would return at once and nothing is waiting on it. + private static func drain(_ output: MLXOutputCollector, _ stdoutPipe: Pipe, _ stderrPipe: Pipe) async { + for _ in 0..<20 { + if output.bothStreamsClosed || Task.isCancelled { break } + try? await Task.sleep(for: .milliseconds(50)) + } + stdoutPipe.fileHandleForReading.readabilityHandler = nil + stderrPipe.fileHandleForReading.readabilityHandler = nil + } + + /// SIGTERM now, SIGKILL three seconds later if the child ignored it. + private static func stop(_ process: Process) { + guard process.isRunning else { return } + process.terminate() + let pid = process.processIdentifier + DispatchQueue.global(qos: .utility).asyncAfter(deadline: .now() + 3) { + if process.isRunning { kill(pid, SIGKILL) } + } + } +} diff --git a/Cotabby/Services/Runtime/MLX/MLXServerLogWriter.swift b/Cotabby/Services/Runtime/MLX/MLXServerLogWriter.swift new file mode 100644 index 000000000..12da08212 --- /dev/null +++ b/Cotabby/Services/Runtime/MLX/MLXServerLogWriter.swift @@ -0,0 +1,90 @@ +import Foundation + +/// File overview: +/// Appends everything `mlx_lm.server` writes to stderr to `~/Library/Logs//mlx-server.log`, +/// so the cause of a server failure survives the in-memory tail. It exists because a generator +/// that died once could not be diagnosed afterwards: the only copy of the traceback was a 20-line +/// buffer in the app. Each server start opens with a header line that says which pid, model and +/// port the lines below belong to. +/// +/// Debug-only, like the JSONL sinks: `MLXLiveProcessLauncher` uses the shared writer only when the +/// app runs with `-cotabby-debug` (`CotabbyDebugOptions.isEnabled`). In a normal run the server's +/// stderr stays in memory and nothing is written here. +/// +/// Owned by `MLXLiveProcessLauncher` (a single shared writer). `write` is called from the pipe's +/// readability handler on a background thread, so the handle is guarded by a lock and nothing here +/// touches the main actor. Rotation is the same one-step scheme as `LLMIOFileWriter`. +nonisolated final class MLXServerLogWriter: @unchecked Sendable { + static let shared = MLXServerLogWriter() + static let defaultSizeCapBytes: UInt64 = 2 * 1024 * 1024 + + private let lock = NSLock() + private let fileURL: URL? + private let sizeCapBytes: UInt64 + private var handle: FileHandle? + private var currentByteOffset: UInt64 = 0 + + /// Tests inject a temp-directory URL and a small cap. + init(fileURL: URL? = nil, sizeCapBytes: UInt64 = MLXServerLogWriter.defaultSizeCapBytes) { + self.fileURL = fileURL ?? Self.defaultFileURL() + self.sizeCapBytes = sizeCapBytes + } + + // Same note as `LLMIOFileWriter`: the target's default MainActor isolation would route + // deallocation through a back-deployment shim that has crashed on macOS 26. + nonisolated deinit {} + + func writeHeader(pid: Int32, model: String, port: Int, date: Date = Date()) { + let stamp = ISO8601DateFormatter().string(from: date) + write(Data("=== \(stamp) server pid=\(pid) model=\(model) port=\(port) ===\n".utf8)) + } + + func write(_ data: Data) { + guard !data.isEmpty else { return } + lock.lock() + defer { lock.unlock() } + if currentByteOffset >= sizeCapBytes { rotateLocked() } + if handle == nil { openLocked() } + guard let handle else { return } + do { + try handle.write(contentsOf: data) + currentByteOffset += UInt64(data.count) + } catch { + // Diagnostics only: a full disk must never disturb the server or the app. + } + } + + /// Renames the current file to `mlx-server.log.1` (replacing an older one) and starts fresh. + private func rotateLocked() { + guard let fileURL else { return } + try? handle?.close() + handle = nil + let rotated = fileURL.appendingPathExtension("1") + let manager = FileManager.default + try? manager.removeItem(at: rotated) + try? manager.moveItem(at: fileURL, to: rotated) + openLocked() + } + + private func openLocked() { + guard let fileURL else { return } + let manager = FileManager.default + try? manager.createDirectory(at: fileURL.deletingLastPathComponent(), withIntermediateDirectories: true) + if !manager.fileExists(atPath: fileURL.path) { manager.createFile(atPath: fileURL.path, contents: nil) } + handle = try? FileHandle(forWritingTo: fileURL) + currentByteOffset = (try? handle?.seekToEnd()) ?? 0 + } + + /// `~/Library/Logs//mlx-server.log`, the folder `FileLogHandler` and + /// `LLMIOFileWriter` use (their helpers are private, so the same lookup is repeated here). + private static func defaultFileURL() -> URL? { + guard let library = FileManager.default.urls(for: .libraryDirectory, in: .userDomainMask).first else { + return nil + } + let bundleName = (Bundle.main.object(forInfoDictionaryKey: "CFBundleName") as? String) ?? "Cotabby" + return library + .appendingPathComponent("Logs", isDirectory: true) + .appendingPathComponent(bundleName, isDirectory: true) + .appendingPathComponent("mlx-server.log") + } +} diff --git a/Cotabby/Services/Runtime/MLX/MLXServerManager.swift b/Cotabby/Services/Runtime/MLX/MLXServerManager.swift new file mode 100644 index 000000000..adb259c8f --- /dev/null +++ b/Cotabby/Services/Runtime/MLX/MLXServerManager.swift @@ -0,0 +1,738 @@ +import Combine +import Darwin +import Foundation +import Logging + +/// File overview: +/// Owns Cotabby's `mlx_lm.server` child process. It launches the server on a free loopback port, +/// waits until the model is loaded, restarts it with backoff after a crash, stops it +/// (SIGTERM, then SIGKILL after 3 s), and cleans up a server orphaned by an earlier crash of +/// Cotabby. It publishes one `MLXServerState` that Settings and the router read. +/// +/// Owned by `CotabbyAppEnvironment` for the app's lifetime. It never decides *whether* MLX should +/// run; `MLXServerDesireResolver` does, and the environment calls `apply(_:)` with the answer. +/// The process, readiness probe, port, clock and sleep are injected so the whole lifecycle is +/// testable without Python. +/// +/// Concurrency: everything runs on the main actor. Every launch bumps `generation`, and so does +/// every intended stop. A readiness answer, exit callback or restart timer that belongs to an +/// older generation is ignored, which is what keeps a slow probe from reviving a stopped server. + +/// Why the server could not be started or kept ready. Its `errorDescription` is what the user sees +/// in the Settings callout, so each message says what happened, not how the code failed. +nonisolated enum MLXServerError: LocalizedError, Equatable { + case noFreePort + case readinessTimedOut + case warmUpFailed(statusCode: Int) + + var errorDescription: String? { + switch self { + case .noFreePort: return "No free local port for MLX." + case .readinessTimedOut: return "The model did not load within 2 minutes." + case .warmUpFailed(let code): return "The first test completion failed (HTTP \(code))." + } + } +} + +/// The pid of the server Cotabby launched, kept on disk so the next launch can find a server +/// orphaned when Cotabby crashed or was force-quit. +nonisolated struct MLXServerPIDFile { + let url: URL + + func write(pid: Int32) { + try? FileManager.default.createDirectory(at: url.deletingLastPathComponent(), withIntermediateDirectories: true) + try? Data("\(pid)\n".utf8).write(to: url, options: .atomic) + } + + func read() -> Int32? { + guard let text = try? String(contentsOf: url, encoding: .utf8) else { return nil } + return Int32(text.trimmingCharacters(in: .whitespacesAndNewlines)) + } + + func remove() { + try? FileManager.default.removeItem(at: url) + } +} + +/// Asks the kernel for a free loopback port by binding 127.0.0.1:0, then releases it for the +/// server. Another process could take it in between; the server then fails to bind, exits, and +/// the restart picks a new port. +nonisolated enum MLXLoopbackPort { + static func next() throws -> Int { + let socketFD = socket(AF_INET, SOCK_STREAM, 0) + guard socketFD >= 0 else { throw MLXServerError.noFreePort } + defer { close(socketFD) } + var address = sockaddr_in() + address.sin_len = UInt8(MemoryLayout.size) + address.sin_family = sa_family_t(AF_INET) + address.sin_port = 0 + address.sin_addr.s_addr = inet_addr("127.0.0.1") + var length = socklen_t(MemoryLayout.size) + let bound = withUnsafeMutablePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { bind(socketFD, $0, length) } + } + guard bound == 0 else { throw MLXServerError.noFreePort } + let named = withUnsafeMutablePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(socketFD, $0, &length) } + } + guard named == 0 else { throw MLXServerError.noFreePort } + return Int(UInt16(bigEndian: address.sin_port)) + } +} + +/// Reads another process's state through BSD calls; used only for orphan cleanup. A caseless enum +/// because it holds no state: each call is one `kill` or `sysctl` probe of a pid. +nonisolated enum MLXProcessInspector { + static func isAlive(_ pid: Int32) -> Bool { + kill(pid, 0) == 0 + } + + /// The process's argv via `sysctl(KERN_PROCARGS2)`; nil if it exited or is not ours to read. + static func arguments(of pid: Int32) -> [String]? { + var mib: [Int32] = [CTL_KERN, KERN_PROCARGS2, pid] + var size = 0 + guard sysctl(&mib, 3, nil, &size, nil, 0) == 0, size > 0 else { return nil } + var buffer = [UInt8](repeating: 0, count: size) + guard sysctl(&mib, 3, &buffer, &size, nil, 0) == 0 else { return nil } + return MLXToolchainRules.parseProcArgs(Array(buffer.prefix(size))) + } + + /// True when `pid` has a TCP socket whose local port is `port`. libproc lists a process's file + /// descriptors; each socket descriptor is then asked for its details. Kernel ports come back in + /// network byte order, so they are swapped before comparing. + static func ownsTCPSocket(pid: Int32, port: Int) -> Bool { + // First call sizes the list, second call fills it. The fd count can change in between, so + // only the entries the second call reports are read. + let listSize = proc_pidinfo(pid, PROC_PIDLISTFDS, 0, nil, 0) + guard listSize > 0 else { return false } + var descriptors = [proc_fdinfo]( + repeating: proc_fdinfo(), count: Int(listSize) / MemoryLayout.stride) + let filled = proc_pidinfo(pid, PROC_PIDLISTFDS, 0, &descriptors, listSize) + guard filled > 0 else { return false } + let count = Int(filled) / MemoryLayout.stride + for descriptor in descriptors.prefix(count) where descriptor.proc_fdtype == UInt32(PROX_FDTYPE_SOCKET) { + var info = socket_fdinfo() + let size = Int32(MemoryLayout.size) + // A descriptor can close between the list and this call; a short read means no answer. + guard proc_pidfdinfo(pid, descriptor.proc_fd, PROC_PIDFDSOCKETINFO, &info, size) == size, + info.psi.soi_kind == SOCKINFO_TCP + else { continue } + let localPort = Int(UInt16(bigEndian: UInt16(truncatingIfNeeded: info.psi.soi_proto.pri_tcp.tcpsi_ini.insi_lport))) + if localPort == port { return true } + } + return false + } +} + +/// The state machine around one `mlx_lm.server` child. It is owned by `CotabbyAppEnvironment` (one +/// per app) and collaborates with the resolver (what to run), the launcher and probe (how to run +/// and check it, injected so tests use fakes), and the pid file (orphan recovery). Views and the +/// router only read its published `state`. +@MainActor +final class MLXServerManager: ObservableObject { + typealias Sleep = (Duration) async throws -> Void + + static let restartBackoff: [Duration] = [.seconds(1), .seconds(5), .seconds(30)] + /// Failure policy: restart after 1 s, 5 s, 30 s; give up (`failed`) after `failureLimit` + /// failures inside `failureWindow`, or after `failureLimit` consecutive failures regardless + /// of timing (a server that takes 60 s+ to fail each time never fills the window). + static let failureWindow: TimeInterval = 120 + static let failureLimit = 3 + static let readinessTimeout: Duration = .seconds(120) + static let stopGrace: Duration = .seconds(3) + private static let stopPoll: Duration = .milliseconds(100) + + @Published private(set) var state: MLXServerState = .idle + + private struct RunningServer { + let launch: MLXServerLaunch + let port: Int + let process: any MLXRunningProcess + } + + private let launcher: any MLXProcessLaunching + private let probe: any MLXReadinessProbing + private let pickPort: () throws -> Int + private let sleep: Sleep + private let now: () -> Date + private let pidFile: MLXServerPIDFile + private let baseEnvironment: [String: String] + /// Answers "does this pid own a TCP socket on this port?" before `.ready` is published. The + /// port was only probed, not reserved, so another local program may have taken it. + private let verifyListener: (Int32, Int) -> Bool + + private var desired: MLXServerDesire = .stopped + private var current: RunningServer? + /// The live server's pid, for the Performance pane's resource graph. Nil when none is running. + var runningProcessIdentifier: Int32? { current?.process.processIdentifier } + /// The process a stop is waiting on right now (inside its SIGTERM grace). `current` is already + /// nil by then, so `shutdownSync` needs this reference to still reach it. + private var stoppingProcess: (any MLXRunningProcess)? + private var generation: UInt64 = 0 + private var readinessTask: Task? + private var restartTask: Task? + /// Failure times inside the two-minute window (decides `failed`). + private var recentFailures: [Date] = [] + /// Failures since the last time the server became ready (decides the backoff step). + private var consecutiveFailures = 0 + + init( + paths: MLXPaths, + launcher: any MLXProcessLaunching, + probe: any MLXReadinessProbing, + pickPort: @escaping () throws -> Int = MLXLoopbackPort.next, + sleep: @escaping Sleep = { try await Task.sleep(for: $0) }, + now: @escaping () -> Date = Date.init, + baseEnvironment: [String: String] = ProcessInfo.processInfo.environment, + verifyListener: @escaping (Int32, Int) -> Bool = MLXProcessInspector.ownsTCPSocket(pid:port:) + ) { + self.launcher = launcher + self.probe = probe + self.pickPort = pickPort + self.sleep = sleep + self.now = now + self.pidFile = MLXServerPIDFile(url: paths.serverPIDFile) + self.baseEnvironment = baseEnvironment + self.verifyListener = verifyListener + } + + static func live(paths: MLXPaths) -> MLXServerManager { + MLXServerManager(paths: paths, launcher: MLXLiveProcessLauncher(), probe: MLXHTTPReadinessProbe()) + } + + /// The endpoint the MLX engine instance talks to. Only a ready server yields one; otherwise + /// the reason surfaces as the request's `.unavailable` message (no fallback engine). + var readyConfiguration: OpenAICompatibleEndpointConfiguration { + get throws { + guard case .ready(let port, let modelID) = state else { + throw SuggestionClientError.unavailable(state.notReadyMessage) + } + return try OpenAICompatibleEndpointConfiguration( + baseURLString: "http://127.0.0.1:\(port)/v1", + modelName: modelID, + apiMode: .completions + ) + } + } + + /// Moves toward `desire`. Idempotent: the same desire again changes nothing (a failed server + /// stays failed until `retry()` or a different desire). + func apply(_ desire: MLXServerDesire) async { + guard desire != desired else { return } + desired = desire + recentFailures = [] + consecutiveFailures = 0 + await stopCurrent() + // Another apply may have arrived while the old server was stopping; the newest wins. + guard desired == desire else { return } + switch desire { + case .stopped: + state = .idle + case .unavailable(let reason): + state = .unavailable(reason) + case .running(let launch): + start(launch) + } + } + + func retry() async { + guard case .running(let launch) = desired else { return } + recentFailures = [] + consecutiveFailures = 0 + await stopCurrent() + guard desired == .running(launch) else { return } + start(launch) + } + + /// Blocking stop for `applicationWillTerminate`, where no run loop turn is left for `async`. + func shutdownSync(timeoutSeconds: TimeInterval = 3) { + generation &+= 1 + readinessTask?.cancel() + restartTask?.cancel() + desired = .stopped + defer { + pidFile.remove() + state = .idle + } + // A server already being stopped has had its SIGTERM sent; it only needs the wait and the + // SIGKILL fallback, so it is not terminated a second time. + let stopping = stoppingProcess + let running = current?.process + current = nil + running?.terminate() + let deadline = Date().addingTimeInterval(timeoutSeconds) + while [running, stopping].contains(where: { $0?.isRunning == true }), Date() < deadline { + usleep(50_000) + } + for process in [running, stopping] { + if let process, process.isRunning { process.forceKill() } + } + } + + /// Terminates a server left running by an earlier Cotabby that crashed or was force-quit. + /// Only a process whose live command line is our loopback `mlx_lm.server` is signalled, so a + /// reused pid never kills an unrelated program. + func cleanUpOrphan( + isAlive: @escaping (Int32) -> Bool = MLXProcessInspector.isAlive, + arguments: @escaping (Int32) -> [String]? = MLXProcessInspector.arguments(of:), + signal: @escaping (Int32, Int32) -> Void = { pid, signal in _ = kill(pid, signal) } + ) async { + guard let pid = pidFile.read() else { return } + // The file names the live server when one is running; that record belongs to it, not to an orphan. + guard current?.process.processIdentifier != pid else { return } + pidFile.remove() + guard isAlive(pid), + let processArguments = arguments(pid), + MLXToolchainRules.isOurServer(arguments: processArguments) + else { return } + CotabbyLogger.runtime.warning("Stopping an MLX server left by an earlier Cotabby run", metadata: ["pid": .stringConvertible(pid)]) + signal(pid, SIGTERM) + var waited: Duration = .zero + while isAlive(pid), waited < Self.stopGrace { + try? await sleep(Self.stopPoll) + waited += Self.stopPoll + } + if isAlive(pid) { signal(pid, SIGKILL) } + } + + // MARK: - Lifecycle + + private func start(_ launch: MLXServerLaunch) { + generation &+= 1 + let launchGeneration = generation + state = .starting(modelID: launch.modelID) + let port: Int + let process: any MLXRunningProcess + do { + port = try pickPort() + process = try launcher.launch( + executable: launch.executable, + arguments: MLXToolchainRules.serverArguments( + modelID: launch.modelID, port: port, promptCacheBytes: launch.promptCacheBytes), + environment: MLXToolchainRules.serverEnvironment(base: baseEnvironment), + onExit: { [weak self] status in + self?.processExited(generation: launchGeneration, status: status) + } + ) + } catch { + recordFailure(launch, reason: "MLX could not start: \(error.localizedDescription)", logTail: []) + return + } + current = RunningServer(launch: launch, port: port, process: process) + pidFile.write(pid: process.processIdentifier) + CotabbyLogger.runtime.info("MLX server launched", metadata: [ + "model": .string(launch.modelID), + "port": .stringConvertible(port), + "pid": .stringConvertible(process.processIdentifier), + "prompt_cache_bytes": .stringConvertible(launch.promptCacheBytes) + ]) + let startedAt = now() + readinessTask = Task { [weak self, probe] in + do { + try await probe.waitUntilReady(port: port, modelID: launch.modelID, timeout: Self.readinessTimeout) + // A crash or stop can land while the probe is suspended. Its exit clears `current` + // without bumping `generation`, so the identity check is what stops a dead server + // from being published as ready. + guard let self, self.generation == launchGeneration, !Task.isCancelled, + self.current?.process === process else { return } + guard self.verifyListener(process.processIdentifier, port) else { + await self.failCurrent( + generation: launchGeneration, + reason: "Another program answered on MLX's port." + ) + return + } + self.consecutiveFailures = 0 + self.state = .ready(port: port, modelID: launch.modelID) + CotabbyLogger.runtime.info("MLX server ready", metadata: [ + "model": .string(launch.modelID), + "ready_ms": .stringConvertible(Int(self.now().timeIntervalSince(startedAt) * 1000)) + ]) + } catch { + guard let self, self.generation == launchGeneration, !(error is CancellationError) else { return } + await self.failCurrent( + generation: launchGeneration, + reason: "MLX did not become ready: \(error.localizedDescription)" + ) + } + } + } + + /// An exit nobody asked for. Intended stops bump `generation` or clear `current` first, so + /// their exits land here and are ignored. + private func processExited(generation exitGeneration: UInt64, status: Int32) { + guard exitGeneration == generation, let running = current else { return } + current = nil + readinessTask?.cancel() + readinessTask = nil + pidFile.remove() + CotabbyLogger.runtime.error("MLX server exited", metadata: ["status": .stringConvertible(status)]) + recordFailure(running.launch, reason: "MLX stopped unexpectedly (exit status \(status)).", + logTail: running.process.stderrTail()) + } + + /// A request showed that the server's generator died while the process stayed up (mlx_lm answers + /// 404 "generation thread died" and `/v1/models` keeps working), so the exit callback never + /// fires. This restarts it through the normal failure path: backoff 1 s / 5 s / 30 s, and + /// `failed` after three. Only a report for the live, ready server's port counts; a late report + /// from a request to an old port, or one made while the server is already restarting, is + /// ignored. Dozens of in-flight requests fail together, so everything that must happen once + /// (clearing `current`, leaving `.ready`) happens synchronously before the first suspension. + func reportBroken(port: Int, reason: String) { + guard let running = current, running.port == port, case .ready = state else { return } + let failedGeneration = generation + let tail = running.process.stderrTail() + // Clearing `current` makes the process's own exit callback a no-op; `.starting` makes new + // requests fail fast with "loading" instead of hitting the broken server. SIGTERM goes out + // and the process is recorded as stopping before the task below is scheduled, so a quit + // that lands first still finds it in `shutdownSync` (which then waits and SIGKILLs). + current = nil + state = .starting(modelID: running.launch.modelID) + stoppingProcess = running.process + running.process.terminate() + CotabbyLogger.runtime.error("MLX generator stopped working", metadata: [ + "reason": .string(reason), + "port": .stringConvertible(port), + "log_tail": .string(tail.joined(separator: "\n")) + ]) + Task { [weak self] in + guard let self else { return } + await self.waitForExitTracked(running.process) + self.removePIDFile(ownedBy: running.process.processIdentifier) + // A stop, model change or shutdown can land during the SIGTERM grace; it owns the + // state then, so no failure is recorded for a server the user no longer wants. + guard self.generation == failedGeneration, self.desired == .running(running.launch) else { return } + self.recordFailure(running.launch, reason: "MLX's generator stopped working: \(reason)", logTail: tail) + } + } + + /// Readiness failed: the process is still up but useless. Kill it, then count a failure. + private func failCurrent(generation failedGeneration: UInt64, reason: String) async { + guard let running = current else { return } + let tail = running.process.stderrTail() + current = nil + await terminateTracked(running.process) + removePIDFile(ownedBy: running.process.processIdentifier) + guard generation == failedGeneration, desired == .running(running.launch) else { return } + recordFailure(running.launch, reason: reason, logTail: tail) + } + + private func recordFailure(_ launch: MLXServerLaunch, reason: String, logTail: [String]) { + let at = now() + recentFailures = recentFailures.filter { at.timeIntervalSince($0) < Self.failureWindow } + [at] + consecutiveFailures += 1 + CotabbyLogger.runtime.error("MLX server failure", metadata: [ + "reason": .string(reason), + "failures_in_window": .stringConvertible(recentFailures.count) + ]) + // Two ways to give up: a burst (limit failures inside the window) or a slow loop (limit + // failures in a row without ever becoming ready, each attempt outlasting the window). + guard recentFailures.count < Self.failureLimit, consecutiveFailures < Self.failureLimit else { + generation &+= 1 + state = .failed(reason: reason, logTail: logTail) + return + } + let delay = Self.restartBackoff[min(consecutiveFailures - 1, Self.restartBackoff.count - 1)] + let restartGeneration = generation + state = .starting(modelID: launch.modelID) + restartTask = Task { [weak self] in + guard let self else { return } + do { try await self.sleep(delay) } catch { return } + guard self.generation == restartGeneration, self.desired == .running(launch) else { return } + self.restartTask = nil + self.start(launch) + } + } + + /// Stops whatever runs now. Bumping `generation` first invalidates any pending readiness, + /// restart or exit callback that belongs to it. + private func stopCurrent() async { + generation &+= 1 + readinessTask?.cancel() + readinessTask = nil + restartTask?.cancel() + restartTask = nil + guard let running = current else { + pidFile.remove() + return + } + current = nil + state = .stopping + await terminateTracked(running.process) + removePIDFile(ownedBy: running.process.processIdentifier) + CotabbyLogger.runtime.info("MLX server stopped", metadata: ["model": .string(running.launch.modelID)]) + } + + /// Sends SIGTERM and waits for the exit while keeping the process visible to `shutdownSync`. + private func terminateTracked(_ process: any MLXRunningProcess) async { + stoppingProcess = process + process.terminate() + await waitForExitTracked(process) + } + + /// The second half of a stop, for a process that has already had its SIGTERM: waits out the + /// grace period, SIGKILLs it if needed, then stops tracking it. + private func waitForExitTracked(_ process: any MLXRunningProcess) async { + await Self.waitForExit(process, sleep: sleep) + if stoppingProcess === process { stoppingProcess = nil } + } + + /// Removes the pid file only while it still names `pid`. A stop waits up to 3 s, and a newer + /// `start` may have written its own pid into the same file during that wait. + private func removePIDFile(ownedBy pid: Int32) { + if pidFile.read() == pid { pidFile.remove() } + } + + private static func waitForExit(_ process: any MLXRunningProcess, sleep: Sleep) async { + var waited: Duration = .zero + while process.isRunning, waited < stopGrace { + try? await sleep(stopPoll) + waited += stopPoll + } + if process.isRunning { process.forceKill() } + } +} + +// MARK: - Live process and probe + +/// Starts the real `mlx_lm.server` with stdout discarded. Its stderr always feeds an in-memory tail +/// for the failure message. Under `-cotabby-debug` the full stream is also appended to +/// `mlx-server.log`, like the app's other on-disk logs, so a failure can be diagnosed afterwards; +/// a normal run writes nothing to disk. +final class MLXLiveProcessLauncher: MLXProcessLaunching { + private let logWriter: MLXServerLogWriter? + + init(logWriter: MLXServerLogWriter? = CotabbyDebugOptions.isEnabled ? .shared : nil) { + self.logWriter = logWriter + } + + func launch( + executable: URL, + arguments: [String], + environment: [String: String], + onExit: @escaping @MainActor (Int32) -> Void + ) throws -> any MLXRunningProcess { + // The exit arrives on a background thread; hop to the main actor for the manager. + let stderr = Pipe() + // Raw lines are kept generously (300) because the filter in `stderrTail()` drops + // access-log lines and disconnect tracebacks, which are most of what the server prints. + let output = MLXOutputCollector(tailLines: 300) + let process = Self.configuredProcess( + executable: executable, + arguments: arguments, + environment: environment, + stderr: stderr, + onExit: { @Sendable status in + Task { @MainActor in onExit(status) } + } + ) + try process.run() + // The header needs the pid, so the reader starts after `run()`; the pipe buffers whatever + // the server wrote in between. + logWriter?.writeHeader( + pid: process.processIdentifier, + model: Self.value(after: "--model", in: arguments) ?? "?", + port: Int(Self.value(after: "--port", in: arguments) ?? "") ?? 0) + Self.startReading(stderr, into: output, log: logWriter) + return MLXLiveRunningProcess(process: process, output: output) + } + + /// Builds the process outside the main actor. Its handlers run on background threads, and + /// with the target's default MainActor isolation a closure formed in a main-actor method could + /// be inferred main-actor-isolated and trip Swift's runtime isolation check there. + nonisolated private static func configuredProcess( + executable: URL, + arguments: [String], + environment: [String: String], + stderr: Pipe, + onExit: @escaping @Sendable (Int32) -> Void + ) -> Process { + let process = Process() + process.executableURL = executable + process.arguments = arguments + process.environment = environment + process.standardInput = FileHandle.nullDevice + process.standardOutput = FileHandle.nullDevice + process.standardError = stderr + process.terminationHandler = { finished in onExit(finished.terminationStatus) } + return process + } + + /// Drains stderr off the main actor: each chunk goes to the in-memory collector and, when one + /// is given, the log file. + /// At EOF the handler is called with empty data, and it keeps being called until it is removed. + /// Removing it there and recording the close matches `MLXProcessRunner`. + nonisolated private static func startReading(_ pipe: Pipe, into output: MLXOutputCollector, log: MLXServerLogWriter?) { + pipe.fileHandleForReading.readabilityHandler = { handle in + let data = handle.availableData + if data.isEmpty { + handle.readabilityHandler = nil + output.markStderrClosed() + } else { + output.appendStderr(data) + log?.write(data) + } + } + } + + nonisolated private static func value(after flag: String, in arguments: [String]) -> String? { + guard let index = arguments.firstIndex(of: flag), index + 1 < arguments.count else { return nil } + return arguments[index + 1] + } +} + +/// Adapts a Foundation `Process` to `MLXRunningProcess`. It is created by +/// `MLXLiveProcessLauncher` and owned by the manager's `RunningServer`; it lives exactly as long as +/// that launch. It holds the stderr collector so the failure message can quote the server's last +/// lines even after the process has exited. +final class MLXLiveRunningProcess: MLXRunningProcess { + private let process: Process + private let output: MLXOutputCollector + + init(process: Process, output: MLXOutputCollector) { + self.process = process + self.output = output + } + + var processIdentifier: Int32 { process.processIdentifier } + var isRunning: Bool { process.isRunning } + + func terminate() { + if process.isRunning { process.terminate() } + } + + func forceKill() { + if process.isRunning { kill(process.processIdentifier, SIGKILL) } + } + + func stderrTail() -> [String] { MLXServerLogFilter.relevantTail(output.stderrTail, limit: 20) } +} + +/// Polls `/v1/models` every 250 ms, then sends two streamed warm-up completions shaped like real +/// suggestions for the exact model id, so the weights and Metal kernels load now, not on the +/// user's first keystroke (about 6 s for Gemma 4 E4B). +nonisolated struct MLXHTTPReadinessProbe: MLXReadinessProbing { + func waitUntilReady(port: Int, modelID: String, timeout: Duration) async throws { + let deadline = ContinuousClock.now + timeout + let base = URL(string: "http://127.0.0.1:\(port)/v1")! + let configuration = URLSessionConfiguration.ephemeral + configuration.timeoutIntervalForRequest = 2 + let session = URLSession(configuration: configuration) + defer { session.finishTasksAndInvalidate() } + + while true { + try Task.checkCancellation() + if let (_, response) = try? await session.data(from: base.appendingPathComponent("models")), + (response as? HTTPURLResponse)?.statusCode == 200 { + break + } + guard ContinuousClock.now < deadline else { throw MLXServerError.readinessTimedOut } + try await Task.sleep(for: .milliseconds(250)) + } + + // Two streamed warm-ups instead of one tiny blocking request. MLX builds its Metal kernels + // lazily, per tensor shape and per op, and the stop-word state machine is keyed to the + // request shape. A "Hello" with `stream: false` warms a different path than the first real + // suggestion (streamed, several hundred prompt tokens, `stop` set), so the first user-visible + // completion still paid that cost. The first pass uses the real sampling settings with the + // newline stop word; the second drops `stop` to cover the multi-line path. + try await warmUp(session: session, base: base, modelID: modelID, stop: ["\n"], deadline: deadline) + try await warmUp(session: session, base: base, modelID: modelID, stop: nil, deadline: deadline) + } + + /// A fixed, realistic draft in the shape Cotabby sends: a conditioning preface line, a blank + /// line, then prose that ends mid-sentence. Its length (about 1,900 characters) is what makes + /// the warm-up prefill match a real request. + private static let warmUpPrompt = """ + Format: email; Title: Project status and next steps + + Hi Dana, + + Thanks for sending the revised timeline on Monday. I went through it line by line with the team this \ + morning, and most of the dates hold up well against what we are seeing in the build pipeline. The \ + integration work for the storage layer finished ahead of schedule, which gives us a little room at the \ + end of the month for the reviewer feedback that we always seem to collect late. I still have concerns \ + about the migration step, though. The current plan assumes that every customer account can be moved in a \ + single window on the weekend, but the account export we ran last Thursday took almost twice as long as \ + we estimated, and it only covered about a third of the records. If the remaining records scale the same \ + way, the window will not be long enough, and we would have to split the work across two weekends. That \ + is not a disaster, but it changes who needs to be on call and how we communicate the maintenance notice \ + to customers, so I would like to settle it before we commit to the calendar invitations. Ibrahim has \ + offered to rerun the export with the new indexes enabled, and I think his numbers will tell us quickly \ + whether the single window is realistic. In the meantime, I would suggest we keep the draft schedule as \ + it is, mark the migration as a risk in the tracker, and avoid sending anything to the customer success \ + team until we have the new figures. I know this is not the answer you were hoping for on a Monday \ + morning, and I appreciate that you have been patient while we worked through the details. If it helps, I \ + can join a short call tomorrow to walk through the export log together and agree on the fallback plan \ + before the end of the week, so that everyone leaves the meeting with the same picture of what happens \ + next and who owns each + """ + + /// Sends one streamed completion shaped like a real suggestion and reads it to `[DONE]` or the + /// end of the stream. The sampling values are the ones `SuggestionConfiguration.standard` sends, + /// so the server takes the same sampler path. The request timeout is whatever remains of the + /// readiness deadline, so a slow warm-up cannot outlive the readiness budget. + private func warmUp( + session: URLSession, + base: URL, + modelID: String, + stop: [String]?, + deadline: ContinuousClock.Instant + ) async throws { + var body: [String: Any] = [ + "model": modelID, + "prompt": Self.warmUpPrompt, + "max_tokens": 8, + "stream": true, + "temperature": 0.1, + "top_p": 0.7, + "top_k": 20, + "min_p": 0.08, + "repetition_penalty": 1.025 + ] + if let stop { body["stop"] = stop } + + var request = URLRequest(url: base.appendingPathComponent("completions")) + request.timeoutInterval = Self.remainingSeconds(until: deadline) + request.httpMethod = "POST" + request.setValue("application/json", forHTTPHeaderField: "Content-Type") + request.httpBody = try JSONSerialization.data(withJSONObject: body) + let (bytes, response) = try await session.bytes(for: request) + let status = (response as? HTTPURLResponse)?.statusCode ?? 0 + guard status == 200 else { throw MLXServerError.warmUpFailed(statusCode: status) } + // The server finishes generation only once the client has read the stream, so drain it. + for try await line in bytes.lines where line == "data: [DONE]" { + break + } + } + + /// Seconds left until `deadline`, floored at one second so a request is never sent with a zero + /// timeout, which URLSession would treat as "no timeout" in some configurations. + private static func remainingSeconds(until deadline: ContinuousClock.Instant) -> TimeInterval { + let remaining = deadline - ContinuousClock.now + let seconds = Double(remaining.components.seconds) + Double(remaining.components.attoseconds) / 1e18 + return max(1, seconds) + } +} + +extension OpenAICompatibleSuggestionEngine { + /// The MLX engine: the shared OpenAI-compatible client pointed at Cotabby's own server. The + /// configuration is read per request, so a request made while the server is loading or + /// failed throws `.unavailable` with the reason at once: no HTTP, and no fallback engine. + static func mlx(client: OpenAICompatibleAPIClient, serverManager: MLXServerManager) -> OpenAICompatibleSuggestionEngine { + OpenAICompatibleSuggestionEngine( + client: client, + configurationProvider: { [weak serverManager] in + guard let serverManager else { + throw SuggestionClientError.unavailable("MLX is not running.") + } + return try serverManager.readyConfiguration + }, + apiKeyProvider: { nil }, + backend: .mlx, + onServerBroken: { [weak serverManager] port, reason in + serverManager?.reportBroken(port: port, reason: reason) + } + ) + } +} diff --git a/Cotabby/Services/Runtime/MLX/MLXToolchainInstaller.swift b/Cotabby/Services/Runtime/MLX/MLXToolchainInstaller.swift new file mode 100644 index 000000000..515ef0ca6 --- /dev/null +++ b/Cotabby/Services/Runtime/MLX/MLXToolchainInstaller.swift @@ -0,0 +1,175 @@ +import CryptoKit +import Foundation +import Logging +import Security + +/// File overview: +/// Installs mlx-lm for Cotabby in two verified steps: +/// 1. Download Astral's `uv`, pinned by version and SHA-256, and check its code signature before +/// it ever runs. +/// 2. Have uv install the pinned mlx-lm, with its own Python, entirely under Cotabby's MLX folder. +/// Nothing touches the system Python or the user's shell setup. The network, process and +/// signature steps are injected closures, so tests run the whole sequence without downloading or +/// executing anything. + +nonisolated enum MLXInstallError: LocalizedError, Equatable { + case downloadFailed(String) + case checksumMismatch + case extractFailed + case signatureInvalid + case installFailed(tail: [String]) + case timedOut + + var errorDescription: String? { + switch self { + case .downloadFailed(let message): return "Could not download the MLX installer: \(message)" + case .checksumMismatch: return "The downloaded installer did not match its published checksum, so it was not used." + case .extractFailed: return "Could not unpack the MLX installer." + case .signatureInvalid: return "The installer is not signed by its publisher (Astral), so it was not run." + case .installFailed: return "Installing MLX failed." + case .timedOut: return "Installing MLX took longer than 10 minutes and was stopped." + } + } + + var logTail: [String] { + if case .installFailed(let tail) = self { return tail } + return [] + } +} + +/// The seam `MLXToolchainProvisioner` installs through; tests substitute a fake. +protocol MLXToolchainInstalling { + func install(phase: @escaping @MainActor (MLXInstallPhase) -> Void) async throws +} + +/// One install run. A value type with injected effects; the provisioner owns one for the app's +/// lifetime and calls it at most once at a time. It is `nonisolated` so the hashing, file writes +/// and signature check run off the main actor; only the `phase` callback hops back to it. +nonisolated struct MLXToolchainInstaller: MLXToolchainInstalling { + typealias Download = (URL) async throws -> Data + typealias RunProcess = (URL, [String], [String: String], Duration) async throws -> MLXProcessResult + typealias VerifySignature = (URL) -> Bool + + static let uvArchiveURL = URL(string: "https://github.com/astral-sh/uv/releases/download/0.12.18/uv-aarch64-apple-darwin.tar.gz")! + static let uvArchiveSHA256 = "cf40e0c6a202190ccd9e0406dcfdd5b2d6668a9a5c779b17948963df32aafe5b" + /// Astral Software Inc.'s Apple Developer team. + static let astralTeamID = "2DC432GLL2" + static let totalTimeout: Duration = .seconds(600) + + let paths: MLXPaths + let expectedSHA256: String + let download: Download + let runProcess: RunProcess + let verifySignature: VerifySignature + + init( + paths: MLXPaths, + expectedSHA256: String = MLXToolchainInstaller.uvArchiveSHA256, + download: @escaping Download, + runProcess: @escaping RunProcess, + verifySignature: @escaping VerifySignature + ) { + self.paths = paths + self.expectedSHA256 = expectedSHA256 + self.download = download + self.runProcess = runProcess + self.verifySignature = verifySignature + } + + static func live(paths: MLXPaths) -> MLXToolchainInstaller { + MLXToolchainInstaller( + paths: paths, + download: { url in + let configuration = URLSessionConfiguration.ephemeral + configuration.timeoutIntervalForResource = 180 + let session = URLSession(configuration: configuration) + defer { session.finishTasksAndInvalidate() } + let (data, response) = try await session.data(from: url) + guard (response as? HTTPURLResponse)?.statusCode == 200 else { + throw MLXInstallError.downloadFailed("HTTP \((response as? HTTPURLResponse)?.statusCode ?? 0)") + } + return data + }, + runProcess: { executable, arguments, environment, timeout in + try await MLXProcessRunner.run( + executable: executable, arguments: arguments, environment: environment, timeout: timeout) + }, + verifySignature: Self.isSignedByAstral + ) + } + + func install(phase: @escaping @MainActor (MLXInstallPhase) -> Void) async throws { + let deadline = ContinuousClock.now + Self.totalTimeout + let fileManager = FileManager.default + + await phase(.downloadingUV) + let archive: Data + do { + archive = try await download(Self.uvArchiveURL) + } catch let error as MLXInstallError { + throw error + } catch is CancellationError { + throw CancellationError() + } catch let error where Task.isCancelled || (error as? URLError)?.code == .cancelled { + // URLSession reports a cancelled task as URLError(.cancelled), not CancellationError. + // Map it, so cancelInstall() reverts to the real state instead of showing a failure. + throw CancellationError() + } catch { + throw MLXInstallError.downloadFailed(error.localizedDescription) + } + + await phase(.verifyingUV) + let digest = SHA256.hash(data: archive).map { String(format: "%02x", $0) }.joined() + guard digest == expectedSHA256 else { throw MLXInstallError.checksumMismatch } + try? fileManager.removeItem(at: paths.uvDirectory) + try fileManager.createDirectory(at: paths.uvDirectory, withIntermediateDirectories: true) + let archiveURL = paths.uvDirectory.appendingPathComponent("uv.tar.gz") + try archive.write(to: archiveURL) + let extract = try await runProcess( + URL(fileURLWithPath: "/usr/bin/tar"), + ["-xzf", archiveURL.path, "-C", paths.uvDirectory.path], + MLXToolchainRules.sanitizedEnvironment(ProcessInfo.processInfo.environment), + Self.remaining(until: deadline) + ) + guard extract.status == 0 else { throw MLXInstallError.extractFailed } + guard verifySignature(paths.uvExecutable) else { throw MLXInstallError.signatureInvalid } + + await phase(.installingMLX) + let result: MLXProcessResult + do { + result = try await runProcess( + paths.uvExecutable, + MLXToolchainRules.installArguments(), + MLXToolchainRules.installEnvironment(paths: paths, base: ProcessInfo.processInfo.environment), + Self.remaining(until: deadline) + ) + } catch MLXProcessRunnerError.timedOut { + throw MLXInstallError.timedOut + } + guard result.status == 0 else { throw MLXInstallError.installFailed(tail: result.stderrTail) } + CotabbyLogger.runtime.info("Installed mlx-lm \(MLXToolchainRules.pinnedVersion) with uv") + } + + private static func remaining(until deadline: ContinuousClock.Instant) throws -> Duration { + let left = deadline - ContinuousClock.now + guard left > .zero else { throw MLXInstallError.timedOut } + return left + } + + /// Full signature check, including the executable's own pages (the binary is small), against + /// a requirement naming Astral's team: an Apple-issued Developer ID certificate whose + /// organizational unit is `2DC432GLL2`. + nonisolated static func isSignedByAstral(_ executable: URL) -> Bool { + var staticCode: SecStaticCode? + guard SecStaticCodeCreateWithPath(executable as CFURL, [], &staticCode) == errSecSuccess, + let staticCode + else { return false } + var requirement: SecRequirement? + let text = "anchor apple generic and certificate leaf[subject.OU] = \"\(astralTeamID)\"" + guard SecRequirementCreateWithString(text as CFString, [], &requirement) == errSecSuccess, + let requirement + else { return false } + return SecStaticCodeCheckValidity(staticCode, SecCSFlags(rawValue: kSecCSCheckAllArchitectures), requirement) + == errSecSuccess + } +} diff --git a/Cotabby/Services/Runtime/MLX/MLXToolchainProvisioner.swift b/Cotabby/Services/Runtime/MLX/MLXToolchainProvisioner.swift new file mode 100644 index 000000000..fbf872b19 --- /dev/null +++ b/Cotabby/Services/Runtime/MLX/MLXToolchainProvisioner.swift @@ -0,0 +1,182 @@ +import Combine +import Foundation +import Logging + +/// File overview: +/// The app's single source of truth for "is mlx-lm usable here, and where". It checks the known +/// install places, probes each candidate's version, and runs at most one install at a time, so +/// Settings and the server wiring observe one `state`. Owned by `CotabbyAppEnvironment` for the +/// app's lifetime. The server manager never installs anything; it only reads `state.toolchain`. +@MainActor +final class MLXToolchainProvisioner: ObservableObject { + typealias VersionProbe = (URL) async -> String? + + @Published private(set) var state: MLXToolchainState = .checking + + private let paths: MLXPaths + private let home: URL + private let fileManager: FileManager + private let probeVersion: VersionProbe + private let installer: MLXToolchainInstalling + /// The install that currently holds the single-flight slot, tagged with an identity. A finishing + /// task clears the slot and publishes its state only while it is still this one, so a task that + /// was superseded (cancelled by `remove()`) cannot overwrite a newer install's state. + private var activeInstall: (id: UUID, task: Task)? + + init( + paths: MLXPaths, + home: URL = FileManager.default.homeDirectoryForCurrentUser, + fileManager: FileManager = .default, + probeVersion: @escaping VersionProbe, + installer: MLXToolchainInstalling + ) { + self.paths = paths + self.home = home + self.fileManager = fileManager + self.probeVersion = probeVersion + self.installer = installer + } + + static func live(paths: MLXPaths) -> MLXToolchainProvisioner { + MLXToolchainProvisioner( + paths: paths, + probeVersion: { python in + guard let result = try? await MLXProcessRunner.run( + executable: python, + arguments: ["-I", "-c", MLXToolchainRules.versionProbeScript], + environment: MLXToolchainRules.sanitizedEnvironment(ProcessInfo.processInfo.environment), + timeout: .seconds(5) + ), result.status == 0 else { return nil } + let version = result.stdout.trimmingCharacters(in: .whitespacesAndNewlines) + return version.isEmpty ? nil : version + }, + installer: MLXToolchainInstaller.live(paths: paths) + ) + } + + /// Re-checks the install places unless an install is running (its own completion re-checks). + /// `.checking` is published only when there is no toolchain yet. While MLX runs, a transient + /// `.checking` would turn the server desire into "stopped" and kill the live server, so a + /// rescan keeps the old state until the new result is known, and publishes it only if changed. + func refresh() async { + guard activeInstall == nil else { return } + if state.toolchain == nil { state = .checking } + let located = await locate() + guard activeInstall == nil else { return } + if located != state { state = located } + } + + /// The rescan every "it may have changed" moment runs (pane opened, MLX selected, Retry): re-detect + /// the toolchain, then re-list the model cache against the toolchain's supported architectures. + func rescan(models downloader: MLXModelDownloader) async { + await refresh() + await downloader.refreshCache(toolchain: state.toolchain) + } + + /// First candidate with a parseable launcher and a version at or above the minimum wins. A + /// candidate whose Python does not answer (a half-written install) is skipped, never "ready". + func locate() async -> MLXToolchainState { + var tooOld: MLXToolchainState? + for candidate in MLXToolchainRules.serverCandidates(paths: paths, home: home) { + guard fileManager.isExecutableFile(atPath: candidate.path), + let python = interpreter(of: candidate), + let version = await probeVersion(python) + else { continue } + if MLXToolchainRules.isVersion(version, atLeast: MLXToolchainRules.minimumVersion) { + return .ready(MLXToolchain( + serverExecutable: candidate, + pythonExecutable: python, + version: version, + isCotabbyInstall: candidate.standardizedFileURL == paths.ownServerExecutable.standardizedFileURL + )) + } + tooOld = tooOld ?? .tooOld(version: version, path: candidate.path) + } + return tooOld ?? .notInstalled + } + + /// Starts Cotabby's own install. A second call while one runs is ignored, so every caller + /// shares the one install. + func install() { + guard activeInstall == nil else { return } + CotabbyLogger.runtime.info("MLX install started") + let id = UUID() + // Annotated: the optional chain would otherwise infer Task. + let task = Task { [weak self] in + await self?.runInstall(id: id) + } + activeInstall = (id: id, task: task) + } + + func cancelInstall() { + activeInstall?.task.cancel() + } + + /// Cancels a running install and waits for it to finish before deleting anything, so no uv + /// process can recreate files under the folder being removed. A second install cannot start + /// meanwhile, because the slot stays taken until the cancelled task clears it. Then deletes + /// Cotabby's own MLX folder (models in the Hugging Face cache are kept) and re-checks: an + /// external install may still be usable. + func remove() async { + let running = activeInstall?.task + running?.cancel() + await running?.value + // Tens of thousands of files: delete off the main actor, whose run loop serves the key taps. + let root = paths.root + let manager = fileManager + await Task.detached(priority: .utility) { + guard manager.fileExists(atPath: root.path) else { return } + do { try manager.removeItem(at: root) } catch { + CotabbyLogger.runtime.error("Removing MLX install failed: \(error.localizedDescription)") + } + }.value + CotabbyLogger.runtime.info("Removed Cotabby's MLX install") + let located = await locate() + // An install started during the re-check owns the state now; do not overwrite it. + guard activeInstall == nil else { return } + state = located + } + + /// One install run, publishing its outcome unless a newer install or a remove has taken over. + /// The slot is cleared last, so a remove() that awaits this task resumes only after the state + /// is final. + private func runInstall(id: UUID) async { + defer { if activeInstall?.id == id { activeInstall = nil } } + do { + try await installer.install { [weak self] phase in + guard let self, self.activeInstall?.id == id else { return } + self.state = .installing(phase) + } + let located = await locate() + guard activeInstall?.id == id else { return } + // Only Cotabby's own install counts. A working external install must not hide a broken own one. + state = located.toolchain?.isCotabbyInstall == true + ? located + : .failed(reason: "MLX was installed but does not start.", logTail: []) + } catch is CancellationError { + let located = await locate() + guard activeInstall?.id == id else { return } + state = located + } catch let error as MLXInstallError { + CotabbyLogger.runtime.error("MLX install failed: \(error.localizedDescription)") + guard activeInstall?.id == id else { return } + state = .failed(reason: error.localizedDescription, logTail: error.logTail) + } catch { + CotabbyLogger.runtime.error("MLX install failed: \(error.localizedDescription)") + guard activeInstall?.id == id else { return } + state = .failed(reason: error.localizedDescription, logTail: []) + } + } + + /// Reads the launcher's first lines through any symlink (uv's bin entries are symlinks). + private func interpreter(of script: URL) -> URL? { + guard let handle = try? FileHandle(forReadingFrom: script.resolvingSymlinksInPath()) else { return nil } + defer { try? handle.close() } + let head = (try? handle.read(upToCount: 1024)) ?? Data() + // Only the first two lines matter. Cutting after them keeps a multi-byte character split + // by the 1 KB read from failing the decode of the whole head. + let lineEnds = head.indices.filter { head[$0] == UInt8(ascii: "\n") }.prefix(2) + let lines = head[..<(lineEnds.last.map { head.index(after: $0) } ?? head.endIndex)] + return MLXToolchainRules.interpreter(fromScript: String(bytes: lines, encoding: .utf8) ?? "") + } +} diff --git a/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClient.swift b/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClient.swift index 5d3575c0c..6830f299e 100644 --- a/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClient.swift +++ b/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClient.swift @@ -6,6 +6,15 @@ nonisolated struct OpenAICompatibleGenerationOptions: Equatable, Sendable { let maxPredictionTokens: Int let temperature: Double let topP: Double + /// MLX-only sampling fields. Nil for every other server, so they never receive unknown keys. + let mlx: MLXCompletionOptions? + + init(maxPredictionTokens: Int, temperature: Double, topP: Double, mlx: MLXCompletionOptions? = nil) { + self.maxPredictionTokens = maxPredictionTokens + self.temperature = temperature + self.topP = topP + self.mlx = mlx + } } enum OpenAICompatibleClientError: LocalizedError, Equatable { @@ -78,7 +87,8 @@ final class OpenAICompatibleAPIClient { apiKey: String?, prompt: String, options: OpenAICompatibleGenerationOptions, - onPartialRawText: (@MainActor (String) -> Void)? + onPartialRawText: (@MainActor (String) -> Void)?, + stopWhen: (@MainActor (String, Int) -> Bool)? = nil ) async throws -> String { guard !configuration.modelName.isEmpty else { throw OpenAICompatibleEndpointError.emptyModelName @@ -104,6 +114,7 @@ final class OpenAICompatibleAPIClient { var accumulated = "" var sawEvent = false + var textEvents = 0 for try await line in bytes.lines { try Task.checkCancellation() switch try OpenAICompatibleSSEDecoder.decode(line, mode: configuration.apiMode) { @@ -114,8 +125,16 @@ final class OpenAICompatibleAPIClient { return accumulated case .text(let text): sawEvent = true + textEvents += 1 accumulated += text onPartialRawText?(accumulated) + // Returning leaves the `for await`, which cancels the URLSession task and closes the + // socket. mlx_lm.server's handler fails on its next write and marks the request + // stopped, and its generation thread drops it at the next token, so an early stop + // here saves server time as well as the wait. + if let stopWhen, stopWhen(accumulated, textEvents) { + return accumulated + } case .error(let message): throw OpenAICompatibleClientError.streamError(message) } @@ -286,6 +305,12 @@ private nonisolated struct CompletionRequest: Encodable { let maxTokens: Int let temperature: Double let topP: Double + // MLX-only fields. Optionals are left out of the JSON when nil (synthesized `encodeIfPresent`). + let topK: Int? + let minP: Double? + let repetitionPenalty: Double? + let seed: Int? + let stop: [String]? init(model: String, prompt: String, options: OpenAICompatibleGenerationOptions) { self.model = model @@ -293,12 +318,20 @@ private nonisolated struct CompletionRequest: Encodable { maxTokens = options.maxPredictionTokens temperature = options.temperature topP = options.topP + topK = options.mlx?.topK + minP = options.mlx?.minP + repetitionPenalty = options.mlx?.repetitionPenalty + seed = options.mlx?.seed + stop = options.mlx.flatMap { $0.stop.isEmpty ? nil : $0.stop } } private enum CodingKeys: String, CodingKey { - case model, prompt, stream, temperature + case model, prompt, stream, temperature, seed, stop case maxTokens = "max_tokens" case topP = "top_p" + case topK = "top_k" + case minP = "min_p" + case repetitionPenalty = "repetition_penalty" } } diff --git a/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngine.swift b/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngine.swift index e2709886d..0994a04d7 100644 --- a/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngine.swift +++ b/Cotabby/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngine.swift @@ -3,23 +3,46 @@ import Logging /// Adapts an OpenAI-compatible SSE stream to Cotabby's backend-independent suggestion contract. /// The HTTP client owns wire parsing; this type owns sampling, normalization, logging, and error -/// mapping so the router never needs endpoint-specific behavior. +/// mapping so the router never needs endpoint-specific behavior. One type serves two backends: the +/// user's endpoint, and the loopback `mlx_lm.server` that the MLX engine runs (`Backend`). @MainActor final class OpenAICompatibleSuggestionEngine: SuggestionGenerating { + /// Which server this instance talks to. The endpoint instance sends only OpenAI's basics. The + /// MLX instance adds mlx_lm.server's sampling fields and ends the stream itself at a sentence + /// end, the way the llama runtime stops decoding (`DecodeStopPolicy`). + enum Backend: Equatable { + case endpoint + case mlx + + var logLabel: String { self == .mlx ? "mlx" : "openai_compatible" } + } + private let client: OpenAICompatibleAPIClient private let configurationProvider: @MainActor () throws -> OpenAICompatibleEndpointConfiguration private let apiKeyProvider: @MainActor () throws -> String? /// Owned for the engine's lifetime so focus-scoped warmup calls share one in-flight model load. private let preloadWorkController = OllamaPreloadWorkController() + private let backend: Backend + /// MLX only: told the port and a short reason when a request shows the server's generator died + /// (see `MLXGeneratorFailure`). The endpoint backend never calls it. + private let onServerBroken: (@MainActor (Int, String) -> Void)? + /// Confirms a suspicious 404 against the server's `/health` before reporting it broken. + private let healthCheck: MLXHealthCheck init( client: OpenAICompatibleAPIClient, configurationProvider: @escaping @MainActor () throws -> OpenAICompatibleEndpointConfiguration, - apiKeyProvider: @escaping @MainActor () throws -> String? + apiKeyProvider: @escaping @MainActor () throws -> String?, + backend: Backend = .endpoint, + onServerBroken: (@MainActor (Int, String) -> Void)? = nil, + healthCheck: @escaping MLXHealthCheck = MLXHealthProbe.check(port:) ) { self.client = client self.configurationProvider = configurationProvider self.apiKeyProvider = apiKeyProvider + self.backend = backend + self.onServerBroken = onServerBroken + self.healthCheck = healthCheck } func generateSuggestion(for request: SuggestionRequest) async throws -> SuggestionResult { @@ -32,11 +55,14 @@ final class OpenAICompatibleSuggestionEngine: SuggestionGenerating { ) async throws -> SuggestionResult { let metadata: Logger.Metadata = [ "request_id": .string(request.requestID), - "engine": .string("openai_compatible") + "engine": .string(backend.logLabel) ] let startTime = Date() + // Set once the configuration resolves, so the catch below knows which server failed. + var serverPort: Int? do { let configuration = try configurationProvider() + serverPort = configuration.baseURL.port let apiKey = try apiKeyProvider() let partialHandler: (@MainActor (String) -> Void)? if let onPartial { @@ -71,9 +97,11 @@ final class OpenAICompatibleSuggestionEngine: SuggestionGenerating { options: OpenAICompatibleGenerationOptions( maxPredictionTokens: request.maxPredictionTokens, temperature: request.temperature, - topP: request.topP + topP: request.topP, + mlx: backend == .mlx ? Self.mlxOptions(for: request) : nil ), - onPartialRawText: partialHandler + onPartialRawText: partialHandler, + stopWhen: backend == .mlx ? Self.earlyStop(for: request) : nil ) try Task.checkCancellation() @@ -115,6 +143,13 @@ final class OpenAICompatibleSuggestionEngine: SuggestionGenerating { } catch let error as URLError where error.code == .cancelled { CotabbyLogger.suggestion.debug("Endpoint request cancelled", metadata: metadata) throw SuggestionClientError.cancelled + } catch let error as OpenAICompatibleClientError + where backend == .mlx && MLXGeneratorFailure.reason(for: error) != nil { + throw await mlxGeneratorFailure(error, serverPort: serverPort, metadata: metadata) + } catch let error as SuggestionClientError { + // The MLX configuration provider reports "server not ready" this way, already phrased + // for the user; wrapping it again would turn it into a generic generation failure. + throw error } catch let error as OpenAICompatibleEndpointError { throw SuggestionClientError.unavailable(error.localizedDescription) } catch { @@ -128,10 +163,61 @@ final class OpenAICompatibleSuggestionEngine: SuggestionGenerating { func resetCachedGenerationContext() async {} + /// The error to throw for a request that looks like MLX's generator died. It looks that way, + /// but mlx_lm answers 404 for per-request failures too, so `/health` decides: healthy means + /// this one request failed and the server stays; otherwise the manager is told to restart it. + private func mlxGeneratorFailure( + _ error: OpenAICompatibleClientError, + serverPort: Int?, + metadata: Logger.Metadata + ) async -> SuggestionClientError { + guard let serverPort, let reason = MLXGeneratorFailure.reason(for: error) else { + return .generationFailed(error.localizedDescription) + } + let health = await healthCheck(serverPort) + // The user kept typing while the probe ran: its answer (a cancelled request looks like + // "unreachable") says nothing about the server, so never restart it for this. + if Task.isCancelled || health == .cancelled { return .cancelled } + guard health != .healthy else { + CotabbyLogger.suggestion.error( + "MLX request failed but the server is healthy: \(reason)", metadata: metadata) + return .generationFailed(error.localizedDescription) + } + CotabbyLogger.suggestion.error("MLX generator failed: \(reason)", metadata: metadata) + onServerBroken?(serverPort, reason) + return .unavailable("MLX is restarting…") + } + + /// Same sampling values the llama runtime uses, so MLX and Open Source condition alike. + private static func mlxOptions(for request: SuggestionRequest) -> MLXCompletionOptions { + MLXCompletionOptions( + topK: request.topK, + minP: request.minP, + repetitionPenalty: request.repetitionPenalty, + seed: request.randomSeed.map(Int.init), + stop: request.isMultiLineEnabled ? [] : ["\n"] + ) + } + + /// The llama runtime's stop rule applied to the streamed text: one SSE text event is about one + /// token, which is what `tokensGenerated` counts for the minimum-token guard. + private static func earlyStop(for request: SuggestionRequest) -> @MainActor (String, Int) -> Bool { + let minimumWords = request.wordRange?.lowWords ?? 0 + return { accumulated, events in + DecodeStopPolicy.verdict( + accumulated: accumulated, + tokensGenerated: events, + minimumWords: minimumWords + ) != nil + } + } + /// Best-effort Ollama cold-start work runs through a request that is independent from the /// coordinator's cancellable prediction task. Typing again can therefore discard stale /// autocomplete work without aborting the model load that future requests need. func prewarm(for _: SuggestionRequest) async { + // The Ollama preload path is endpoint-only; the MLX instance warms up in its server manager. + guard backend == .endpoint else { return } do { let configuration = try configurationProvider() guard configuration.defaultOllamaGenerateURL != nil else { return } diff --git a/Cotabby/Services/Runtime/SuggestionEngineRouter.swift b/Cotabby/Services/Runtime/SuggestionEngineRouter.swift index ff8d3f568..1d96a3707 100644 --- a/Cotabby/Services/Runtime/SuggestionEngineRouter.swift +++ b/Cotabby/Services/Runtime/SuggestionEngineRouter.swift @@ -11,6 +11,7 @@ final class SuggestionEngineRouter { private let foundationModelEngine: any SuggestionGenerating private let llamaEngine: any SuggestionGenerating private let openAICompatibleEngine: any SuggestionGenerating + private let mlxEngine: any SuggestionGenerating private let performanceMetricsStore: PerformanceMetricsStore private let qualityMetricsStore: SuggestionQualityMetricsStore /// Closure that returns the currently selected llama model filename (e.g. `Qwen3-0.6B-Q8_0.gguf`). @@ -18,6 +19,10 @@ final class SuggestionEngineRouter { /// on the concrete runtime type — useful for tests that want to fake the model label. private let llamaModelNameProvider: @MainActor () -> String? private let endpointModelNameProvider: @MainActor () -> String? + /// Closure that returns the MLX model repo id the MLX engine is serving, used to label its + /// performance metrics. Like the llama provider, a closure keeps the router from depending on + /// the MLX service types. + private let mlxModelNameProvider: @MainActor () -> String? init( suggestionSettings: SuggestionSettingsModel, @@ -27,7 +32,9 @@ final class SuggestionEngineRouter { qualityMetricsStore: SuggestionQualityMetricsStore, llamaModelNameProvider: @escaping @MainActor () -> String?, openAICompatibleEngine: (any SuggestionGenerating)? = nil, - endpointModelNameProvider: @escaping @MainActor () -> String? = { nil } + endpointModelNameProvider: @escaping @MainActor () -> String? = { nil }, + mlxEngine: (any SuggestionGenerating)? = nil, + mlxModelNameProvider: @escaping @MainActor () -> String? = { nil } ) { self.suggestionSettings = suggestionSettings self.foundationModelEngine = foundationModelEngine @@ -39,6 +46,10 @@ final class SuggestionEngineRouter { self.qualityMetricsStore = qualityMetricsStore self.llamaModelNameProvider = llamaModelNameProvider self.endpointModelNameProvider = endpointModelNameProvider + self.mlxEngine = mlxEngine ?? UnavailableSuggestionEngine( + message: "Set up MLX in Settings → Engine & Model." + ) + self.mlxModelNameProvider = mlxModelNameProvider } func generateSuggestion(for request: SuggestionRequest) async throws -> SuggestionResult { @@ -113,6 +124,16 @@ final class SuggestionEngineRouter { recordPerformanceMetric(modelName: endpointModelNameProvider() ?? "Local Endpoint", latency: result.latency) recordQualityOutcome(result) return result + case .mlx: + // MLX runs in a 127.0.0.1 server Cotabby launched itself, so typing history stays on + // this Mac: unlike the endpoint branch, nothing is withheld here. When the server is + // not ready the MLX engine throws `.unavailable` with the reason; there is deliberately + // no fallback to another engine. + CotabbyLogger.suggestion.debug("Routing to MLX engine", metadata: metadata) + let result = try await mlxEngine.generateSuggestion(for: request, onPartial: onPartial) + recordPerformanceMetric(modelName: mlxModelNameProvider() ?? "MLX", latency: result.latency) + recordQualityOutcome(result) + return result } } @@ -146,6 +167,8 @@ final class SuggestionEngineRouter { return "llama" case .openAICompatible: return "openai_compatible" + case .mlx: + return "mlx" } } @@ -156,6 +179,7 @@ final class SuggestionEngineRouter { await foundationModelEngine.resetCachedGenerationContext() await llamaEngine.resetCachedGenerationContext() await openAICompatibleEngine.resetCachedGenerationContext() + await mlxEngine.resetCachedGenerationContext() } /// Forwards the warmup hook only to the currently selected engine. The inactive backend has @@ -169,6 +193,8 @@ final class SuggestionEngineRouter { await llamaEngine.prewarm(for: request) case .openAICompatible: await openAICompatibleEngine.prewarm(for: request) + case .mlx: + await mlxEngine.prewarm(for: request) } } diff --git a/Cotabby/Support/Runtime/Hardware/GPUStatisticsReader.swift b/Cotabby/Support/Runtime/Hardware/GPUStatisticsReader.swift index 716d205a3..0474c3929 100644 --- a/Cotabby/Support/Runtime/Hardware/GPUStatisticsReader.swift +++ b/Cotabby/Support/Runtime/Hardware/GPUStatisticsReader.swift @@ -52,6 +52,12 @@ nonisolated enum GPUStatisticsReader { /// property read per GPU client on the Mac (68 clients took about 0.5 ms on an M3 Pro), cheap /// enough for the store's one-second timer on the main thread. static func read(pid: pid_t = getpid()) -> GPUStatisticsReading { + read(pids: [pid]) + } + + /// Like `read(pid:)`, but `processGPUTimeNanoseconds` sums the Metal clients of every pid given + /// (Cotabby plus the MLX server when it runs). + static func read(pids: [pid_t]) -> GPUStatisticsReading { var services: io_iterator_t = 0 guard IOServiceGetMatchingServices( kIOMainPortDefault, IOServiceMatching("IOAccelerator"), &services @@ -70,7 +76,7 @@ nonisolated enum GPUStatisticsReader { if statistics == nil { statistics = property("PerformanceStatistics", of: service) as? [String: Any] } - appUsages += appUsageOfClients(of: service, pid: pid) + appUsages += appUsageOfClients(of: service, pids: pids) } return GPUStatisticsReading( deviceUtilizationPercent: deviceUtilizationPercent(from: statistics), @@ -125,24 +131,28 @@ nonisolated enum GPUStatisticsReader { return min(busySeconds / elapsedSeconds * 100, 100) } + /// True when an `IOUserClientCreator` string ("pid 123, Name") belongs to one of `pids`. The + /// trailing comma stops "pid 12," from matching pid 123. + static func creatorMatches(_ creator: String, pids: [pid_t]) -> Bool { + pids.contains { creator.hasPrefix("pid \($0),") } + } + // MARK: - IORegistry access - /// `AppUsage` arrays of the accelerator's clients that `pid` created, one element per client. - /// A client whose `AppUsage` is missing or not an array of dictionaries contributes `nil`. - private static func appUsageOfClients(of service: io_object_t, pid: pid_t) -> [[[String: Any]]?] { + /// `AppUsage` arrays of the accelerator's clients that one of `pids` created, one element per + /// client. A client whose `AppUsage` is missing or not an array of dictionaries contributes `nil`. + private static func appUsageOfClients(of service: io_object_t, pids: [pid_t]) -> [[[String: Any]]?] { var children: io_iterator_t = 0 guard IORegistryEntryGetChildIterator(service, kIOServicePlane, &children) == KERN_SUCCESS else { return [] } defer { IOObjectRelease(children) } - let creatorPrefix = "pid \(pid)," var usages: [[[String: Any]]?] = [] while case let child = IOIteratorNext(children), child != 0 { defer { IOObjectRelease(child) } - // The trailing comma keeps pid 12 from matching pid 123's clients. guard let creator = property("IOUserClientCreator", of: child) as? String, - creator.hasPrefix(creatorPrefix) + creatorMatches(creator, pids: pids) else { continue } usages.append(property("AppUsage", of: child) as? [[String: Any]]) } @@ -156,3 +166,30 @@ nonisolated enum GPUStatisticsReader { .takeRetainedValue() } } + +/// Turns successive cumulative GPU-time readings of a set of processes into a share, 0-100. +/// Owned by the menu bar sampler (one instance, used from one timer). It exists apart from +/// `GPUStatisticsReader` because it is the only stateful part: the baseline must restart whenever +/// the watched pid set changes (the MLX server starting or stopping), otherwise the summed counter +/// jumps or drops and the growth would read as a spike or as nothing. +nonisolated struct GPUShareTracker { + /// One earlier reading: which processes it summed, their GPU time, and when. + private struct Reading { + let pids: Set + let nanoseconds: UInt64? + let date: Date + } + + private var previous: Reading? + + mutating func update(pids: [pid_t], nanoseconds: UInt64?, at date: Date) -> Double? { + let pidSet = Set(pids) + defer { previous = Reading(pids: pidSet, nanoseconds: nanoseconds, date: date) } + guard let previous, previous.pids == pidSet else { return nil } + return GPUStatisticsReader.processUtilizationPercent( + previousNanoseconds: previous.nanoseconds, + currentNanoseconds: nanoseconds, + elapsedSeconds: date.timeIntervalSince(previous.date) + ).map { min(max($0, 0), 100) } + } +} diff --git a/Cotabby/Support/Runtime/Hardware/ResourceUsageSummary.swift b/Cotabby/Support/Runtime/Hardware/ResourceUsageSummary.swift new file mode 100644 index 000000000..147f2de42 --- /dev/null +++ b/Cotabby/Support/Runtime/Hardware/ResourceUsageSummary.swift @@ -0,0 +1,62 @@ +import Foundation + +/// File overview: +/// The menu bar popover's one-line "what is Cotabby using" readout. This file holds the pure value +/// and its text formatting; `MenuResourceMonitor` produces the values and `MenuBarView` shows the +/// string. Kept pure in `Support/` so every formatting branch is testable without a UI or a Mac. + +/// CPU, memory and GPU use of Cotabby (and the MLX server when it runs) at one moment. +nonisolated struct ResourceUsageSummary: Equatable { + let cpuPercent: Double + let memoryBytes: UInt64 + /// `nil` when the GPU cannot be read, or on the first sample (no earlier reading to compare). + let gpuPercent: Double? + /// True when the numbers also count the MLX server process. + let includesMLXServer: Bool + + /// e.g. `Cotabby uses CPU 34% · RAM 2.5 GB · GPU 12%`, with ` incl. MLX` appended when the + /// server is counted. Memory uses 1024-based units, like the Performance pane's byte formatter. + static func menuLine(_ summary: ResourceUsageSummary) -> String { + let cpu = "CPU \(Int(summary.cpuPercent.rounded()))%" + let gpu = summary.gpuPercent.map { "\(Int($0.rounded()))%" } ?? "—" + var line = "Cotabby uses \(cpu) · RAM \(memoryText(summary.memoryBytes)) · GPU \(gpu)" + if summary.includesMLXServer { line += " incl. MLX" } + return line + } + + static func memoryText(_ bytes: UInt64) -> String { + let megabytes = Double(bytes) / 1_048_576 + if megabytes.rounded() < 1024 { return "\(Int(megabytes.rounded())) MB" } + return String(format: "%.1f GB", Double(bytes) / 1_073_741_824) + } +} + +/// Builds `ResourceUsageSummary` values for the menu bar monitor. Owns what makes two periodic +/// samplers safe side by side: its own CPU baseline (so the Performance pane's sampling cannot reset +/// it) and its own GPU baseline. Used from the monitor's timer only. +nonisolated final class MenuResourceSampler { + private let cpuBaseline = PreviousCPUReadings() + private var gpuTracker = GPUShareTracker() + + /// Forget both baselines. Called when the popover closes so the next opening measures a fresh + /// interval instead of averaging over the whole time the menu was closed. + func reset() { + cpuBaseline.withLock { $0.removeAll() } + gpuTracker = GPUShareTracker() + } + + /// `extraProcessID` is the MLX server's pid when it runs. + func sample(extraProcessID: Int32?) -> ResourceUsageSummary { + let pids = [getpid()] + (extraProcessID.map { [$0] } ?? []) + let resources = SystemResourceSampler.sample( + extraProcessIDs: extraProcessID.map { [$0] } ?? [], baseline: cpuBaseline) + let gpu = GPUStatisticsReader.read(pids: pids) + let share = gpuTracker.update(pids: pids, nanoseconds: gpu.processGPUTimeNanoseconds, at: Date()) + return ResourceUsageSummary( + cpuPercent: resources.cpuPercent, + memoryBytes: resources.footprintBytes, + gpuPercent: share, + includesMLXServer: extraProcessID != nil + ) + } +} diff --git a/Cotabby/Support/Runtime/Hardware/SystemResourceSampler.swift b/Cotabby/Support/Runtime/Hardware/SystemResourceSampler.swift index 2d59d7e3e..1fee7f454 100644 --- a/Cotabby/Support/Runtime/Hardware/SystemResourceSampler.swift +++ b/Cotabby/Support/Runtime/Hardware/SystemResourceSampler.swift @@ -18,14 +18,81 @@ struct SystemResourceSample: Equatable { /// Physical memory footprint in bytes. `phys_footprint` is the same figure Activity Monitor and /// Xcode's memory gauge report, so the graph matches what a debugging contributor sees there. let footprintBytes: UInt64 + + /// True when the numbers above also count a helper process Cotabby runs (the MLX server), so + /// the Performance pane can say so instead of making Cotabby look like it alone uses that much. + /// Set by the caller that chose to include the helper, not by the sampler itself. + var includesMLXServer = false } nonisolated enum SystemResourceSampler { - static func sample() -> SystemResourceSample { - SystemResourceSample( - cpuPercent: currentCPUPercent(), - footprintBytes: currentMemoryFootprint() - ) + /// `extraProcessIDs` are child processes whose CPU and memory are added to the totals (the MLX + /// server does the model work that llama.cpp does inside this process, so leaving it out would + /// make an MLX user's graph read near zero). A pid that no longer exists contributes nothing. + /// + /// `baseline` holds the per-pid CPU readings the next call measures against. The default is one + /// shared cache; a second periodic caller (the menu bar monitor) passes its own instance, because + /// two callers sharing one cache would each reset the other's baseline and read CPU over a + /// shortened, wrong window. + static func sample( + extraProcessIDs: [Int32] = [], + baseline previousCPUTime: PreviousCPUReadings = sharedBaseline + ) -> SystemResourceSample { + var cpu = currentCPUPercent() + var footprint = currentMemoryFootprint() + let now = Date() + var seen = Set() + for pid in extraProcessIDs { + guard let usage = processUsage(pid: pid) else { continue } + seen.insert(pid) + footprint += usage.footprintBytes + if let previous = previousCPUTime.withLock({ $0[pid] }) { + cpu += cpuPercent( + previousTicks: previous.ticks, currentTicks: usage.cpuTicks, + elapsedSeconds: now.timeIntervalSince(previous.date), timebase: machTimebase) + } + previousCPUTime.withLock { $0[pid] = (usage.cpuTicks, now) } + } + // Forget processes that are gone or no longer asked about, so a reused pid cannot be + // measured against a stranger's old reading. + previousCPUTime.withLock { cache in cache = cache.filter { seen.contains($0.key) } } + return SystemResourceSample(cpuPercent: cpu, footprintBytes: footprint) + } + + /// CPU use of one process between two readings, in percent of one core (so it can exceed 100). + /// `ri_user_time + ri_system_time` are Mach absolute-time ticks, which on Apple Silicon are not + /// nanoseconds, so they are scaled by the timebase first. Pure so the conversion is testable. + static func cpuPercent( + previousTicks: UInt64, currentTicks: UInt64, elapsedSeconds: TimeInterval, + timebase: (numer: UInt32, denom: UInt32) + ) -> Double { + guard elapsedSeconds > 0, currentTicks >= previousTicks, timebase.denom > 0 else { return 0 } + let deltaNanoseconds = Double(currentTicks - previousTicks) * Double(timebase.numer) / Double(timebase.denom) + return deltaNanoseconds / (elapsedSeconds * 1_000_000_000) * 100 + } + + /// Default CPU baseline, used by the Performance pane's store. The lock keeps it valid if two + /// threads sample at once; separate callers should still use separate instances. + static let sharedBaseline = PreviousCPUReadings() + + private static let machTimebase: (numer: UInt32, denom: UInt32) = { + var info = mach_timebase_info_data_t() + mach_timebase_info(&info) + return (info.numer, info.denom) + }() + + private static func processUsage(pid: Int32) -> (footprintBytes: UInt64, cpuTicks: UInt64)? { + var info = rusage_info_v4() + // `proc_pid_rusage` fills a caller-provided struct through an untyped `rusage_info_t` + // pointer; the rebind hands it our typed storage. Failure (no such process, not ours to + // inspect) returns nonzero and we report nothing for that pid. + let result = withUnsafeMutablePointer(to: &info) { pointer in + pointer.withMemoryRebound(to: rusage_info_t?.self, capacity: 1) { rebound in + proc_pid_rusage(pid, RUSAGE_INFO_V4, rebound) + } + } + guard result == 0 else { return nil } + return (info.ri_phys_footprint, info.ri_user_time + info.ri_system_time) } private static func currentMemoryFootprint() -> UInt64 { @@ -82,3 +149,16 @@ nonisolated enum SystemResourceSampler { return total } } + +/// A tiny lock-protected dictionary of the previous per-pid CPU readings. A class because a static +/// needs reference storage that two callers can share; it owns nothing else. +nonisolated final class PreviousCPUReadings: @unchecked Sendable { + private let lock = NSLock() + private var storage: [Int32: (ticks: UInt64, date: Date)] = [:] + + func withLock(_ body: (inout [Int32: (ticks: UInt64, date: Date)]) -> T) -> T { + lock.lock() + defer { lock.unlock() } + return body(&storage) + } +} diff --git a/Cotabby/Support/Runtime/LocalRuntimeResidencyPolicy.swift b/Cotabby/Support/Runtime/LocalRuntimeResidencyPolicy.swift index 39d7cc79c..e41ea089b 100644 --- a/Cotabby/Support/Runtime/LocalRuntimeResidencyPolicy.swift +++ b/Cotabby/Support/Runtime/LocalRuntimeResidencyPolicy.swift @@ -25,8 +25,9 @@ enum LocalRuntimeResidencyPolicy { // Both switches must be on: a kept-loaded model is useless when the fallback that // would use it is turned off. return isAppleLanguageFallbackEnabled && keepsFallbackModelLoaded - case .openAICompatible: - // The endpoint runs its own model, so a resident GGUF would only double memory use. + case .openAICompatible, .mlx: + // The endpoint and mlx_lm.server each run their own model, so a resident GGUF would + // only double memory use. return false } } diff --git a/Cotabby/Support/Runtime/MLX/MLXCacheScanner.swift b/Cotabby/Support/Runtime/MLX/MLXCacheScanner.swift new file mode 100644 index 000000000..c2a20751d --- /dev/null +++ b/Cotabby/Support/Runtime/MLX/MLXCacheScanner.swift @@ -0,0 +1,156 @@ +import Foundation + +/// File overview: +/// Reads the Hugging Face cache the way huggingface_hub lays it out and decides which cached repos +/// are models `mlx_lm.server` can run for autocomplete. Pure apart from reading the folder it is +/// given, so tests build fixture trees in a temp directory. Lives in `Support/` because it is a +/// rule ("what counts as a usable MLX model"), not a service with state. +/// +/// Cache layout: `/models----/{blobs/[.incomplete], refs/main, +/// snapshots// -> ../../blobs/}`. +nonisolated enum MLXCacheScanner { + /// Same precedence as huggingface_hub. GUI apps rarely inherit these variables (launchd does + /// not read shell profiles), so the default path is the common case. + static func cacheRoot(environment: [String: String], home: URL) -> URL { + if let hubCache = environment["HF_HUB_CACHE"], !hubCache.isEmpty { + return URL(fileURLWithPath: hubCache, isDirectory: true) + } + if let hfHome = environment["HF_HOME"], !hfHome.isEmpty { + return URL(fileURLWithPath: hfHome, isDirectory: true).appendingPathComponent("hub", isDirectory: true) + } + return home.appendingPathComponent(".cache/huggingface/hub", isDirectory: true) + } + + /// Every usable model, sorted by repo id. `supportedModelTypes` is nil when mlx-lm is not + /// installed yet; the type filter is then skipped (nothing can run anyway, and hiding the + /// user's models would read as "your download vanished"). + static func scan( + cacheRoot: URL, + supportedModelTypes: Set?, + fileManager: FileManager = .default + ) -> [MLXCachedModel] { + scanDetailed(cacheRoot: cacheRoot, supportedModelTypes: supportedModelTypes, fileManager: fileManager).models + } + + /// Like `scan`, but also reports the repos that have a complete snapshot and still fail + /// `MLXModelCompatibility`, so the UI can say why a downloaded model is not offered. A repo with + /// no `refs/main` snapshot, or whose snapshot is incomplete (`isComplete`), is neither listed + /// nor rejected: it is simply not downloaded. + static func scanDetailed( + cacheRoot: URL, + supportedModelTypes: Set?, + fileManager: FileManager = .default + ) -> MLXCacheScanResult { + guard let folders = try? fileManager.contentsOfDirectory(atPath: cacheRoot.path) else { + return MLXCacheScanResult(models: [], rejected: [:]) + } + var models: [MLXCachedModel] = [] + var rejected: [MLXRepoID: MLXModelCompatibility.Reason] = [:] + for folder in folders { + guard let repoID = MLXRepoID(cacheFolderName: folder) else { continue } + let repoURL = cacheRoot.appendingPathComponent(folder, isDirectory: true) + guard let snapshot = currentSnapshot(repoURL, fileManager: fileManager), + let files = try? fileManager.contentsOfDirectory(atPath: snapshot.path), + isComplete(snapshot, files: files, fileManager: fileManager) + else { continue } + let config = readConfig(snapshot.appendingPathComponent("config.json")) + switch MLXModelCompatibility.verdict( + repoName: repoID.name, fileNames: files, config: config, supportedModelTypes: supportedModelTypes + ) { + case .compatible(let modelType, let isInstruct): + models.append(MLXCachedModel( + repoID: repoID, + snapshotURL: snapshot, + sizeBytes: snapshotBytes(snapshot, files: files, fileManager: fileManager), + modelType: modelType, + isInstruct: isInstruct + )) + case .incompatible(.missingConfig), .incompatible(.missingWeights), .incompatible(.missingTokenizer): + // A snapshot with files missing is an interrupted download or another tool's + // placeholder: "not downloaded", not "incompatible". Telling the user the model + // can't run would hide the real fix, which is to download it. + continue + case .incompatible(let reason): + rejected[repoID] = reason + } + } + models.sort { $0.repoID.value.localizedCaseInsensitiveCompare($1.repoID.value) == .orderedAscending } + return MLXCacheScanResult(models: models, rejected: rejected) + } + + /// Forwarded so existing callers keep working; the rule lives in `MLXModelCompatibility`. + static func isInstructName(_ name: String) -> Bool { + MLXModelCompatibility.isInstructName(name) + } + + /// Bytes of a repo's blobs, finished and partial, for download progress. + static func downloadedBytes(repo: MLXRepoID, cacheRoot: URL, fileManager: FileManager = .default) -> Int64 { + let blobs = cacheRoot.appendingPathComponent("\(repo.cacheFolderName)/blobs", isDirectory: true) + guard let names = try? fileManager.contentsOfDirectory(atPath: blobs.path) else { return 0 } + return names.reduce(Int64(0)) { sum, name in + let size = (try? fileManager.attributesOfItem(atPath: blobs.appendingPathComponent(name).path)[.size]) as? NSNumber + return sum + (size?.int64Value ?? 0) + } + } + + static func progress(downloadedBytes: Int64, totalBytes: Int64) -> Double? { + guard totalBytes > 0 else { return nil } + return min(1.0, Double(downloadedBytes) / Double(totalBytes)) + } + + // MARK: - Private + + /// Whether the snapshot holds everything `mlx_lm` will load. huggingface_hub links a file into + /// the snapshot only after its blob is complete, so a download in progress elsewhere in the repo + /// (an `.incomplete` blob for another revision or an unused file) does not matter here. Two + /// things do: every entry must reach a readable file (a deleted blob leaves a dangling link), + /// and a sharded model's `model.safetensors.index.json` must find every shard it names, because + /// a download stopped between files leaves some shards, the config and the tokenizer in place. + static func isComplete(_ snapshot: URL, files: [String], fileManager: FileManager = .default) -> Bool { + for name in files { + let target = snapshot.appendingPathComponent(name).resolvingSymlinksInPath() + guard fileManager.isReadableFile(atPath: target.path) else { return false } + } + let indexName = "model.safetensors.index.json" + guard files.contains(indexName) else { return true } + guard let data = try? Data(contentsOf: snapshot.appendingPathComponent(indexName)), + let index = (try? JSONSerialization.jsonObject(with: data)) as? [String: Any], + let weightMap = index["weight_map"] as? [String: String] + else { return false } + return Set(weightMap.values).isSubset(of: Set(files)) + } + + /// The revision `refs/main` names. `mlx_lm.server` runs offline with the repo id, which + /// huggingface_hub resolves through `refs/main`; a snapshot cached only by commit could not be + /// loaded, so it does not count as downloaded. + private static func currentSnapshot(_ repo: URL, fileManager: FileManager) -> URL? { + guard let revision = try? String(contentsOf: repo.appendingPathComponent("refs/main"), encoding: .utf8) + .trimmingCharacters(in: .whitespacesAndNewlines), + !revision.isEmpty + else { return nil } + let url = repo.appendingPathComponent("snapshots/\(revision)", isDirectory: true) + return fileManager.fileExists(atPath: url.path) ? url : nil + } + + private static func readConfig(_ url: URL) -> [String: Any]? { + guard let data = try? Data(contentsOf: url) else { return nil } + return (try? JSONSerialization.jsonObject(with: data)) as? [String: Any] + } + + /// Snapshot files are symlinks into `blobs/`; sizes are read from the link targets. + private static func snapshotBytes(_ snapshot: URL, files: [String], fileManager: FileManager) -> Int64 { + files.reduce(Int64(0)) { sum, name in + let target = snapshot.appendingPathComponent(name).resolvingSymlinksInPath() + let size = (try? fileManager.attributesOfItem(atPath: target.path)[.size]) as? NSNumber + return sum + (size?.int64Value ?? 0) + } + } +} + +/// The outcome of one cache scan: the models Cotabby can use and why the other complete downloads +/// cannot be used. A plain value produced by `MLXCacheScanner` and published by +/// `MLXModelDownloader`; it owns nothing. +nonisolated struct MLXCacheScanResult: Equatable, Sendable { + let models: [MLXCachedModel] + let rejected: [MLXRepoID: MLXModelCompatibility.Reason] +} diff --git a/Cotabby/Support/Runtime/MLX/MLXGeneratorFailure.swift b/Cotabby/Support/Runtime/MLX/MLXGeneratorFailure.swift new file mode 100644 index 000000000..66e8c1ce1 --- /dev/null +++ b/Cotabby/Support/Runtime/MLX/MLXGeneratorFailure.swift @@ -0,0 +1,24 @@ +import Foundation + +/// File overview: +/// Decides whether a failed request to Cotabby's own loopback `mlx_lm.server` *looks like* a dead +/// generator. mlx_lm 0.32.0 answers HTTP 404 (`{"error": "generation thread died"}`) in that case +/// while the process and `/v1/models` stay up, so process-exit monitoring never notices. But its +/// `handle_completion` also maps any exception from `generate()` (tokenizer or chat-template error, +/// a model-load error) to 404, and it never returns 500. So this is only a suspicion: callers must +/// confirm with `GET /health` (`MLXHealthProbe`, 503 only when the generator is gone) before a +/// restart. +nonisolated enum MLXGeneratorFailure { + /// A short reason when `error` could mean the generator is gone, otherwise nil. + static func reason(for error: Error) -> String? { + guard let clientError = error as? OpenAICompatibleClientError else { return nil } + switch clientError { + case .server(let statusCode) where statusCode == 404: + return "HTTP 404" + case .streamError(let message) where message.contains("generation thread died"): + return "generation thread died" + default: + return nil + } + } +} diff --git a/Cotabby/Support/Runtime/MLX/MLXModelCompatibility.swift b/Cotabby/Support/Runtime/MLX/MLXModelCompatibility.swift new file mode 100644 index 000000000..aa0252dba --- /dev/null +++ b/Cotabby/Support/Runtime/MLX/MLXModelCompatibility.swift @@ -0,0 +1,95 @@ +import Foundation + +/// File overview: +/// The single definition of "can `mlx_lm.server` run this repo as an autocomplete model". Three +/// callers share it so they can never disagree: the cache scanner (which downloaded models to +/// list), the custom-repo check (before any download starts) and the desire resolver (to explain a +/// selected model that was rejected). It is pure, so every rule is pinned by a unit test without a +/// file system, a network or a process. It lives in `Support/` because it is policy, not a service. +/// +/// The caller supplies what it already has: the file names (a cached snapshot's, or a remote repo +/// tree's), the parsed `config.json`, and the model types the installed mlx-lm supports. + +nonisolated enum MLXModelCompatibility { + /// Why a repo cannot be used. Each case maps to one sentence for the UI. + enum Reason: Equatable, Sendable { + case missingConfig + case missingWeights + case missingTokenizer + case unreadableConfig + case embeddingModel + case rerankerModel + case speechModel + case classifierModel + /// The `model_type` is not in the set the installed mlx-lm supports. + case unsupportedArchitecture(String) + + var message: String { + switch self { + case .missingConfig: return "No config.json." + case .missingWeights: return "No MLX weights (.safetensors files)." + case .missingTokenizer: return "No tokenizer files." + case .unreadableConfig: return "config.json is unreadable or has no model_type." + case .embeddingModel: return "Embedding model: it can't write text." + case .rerankerModel: return "Reranker model: it can't write text." + case .speechModel: return "Speech or audio model: it can't write text." + case .classifierModel: return "Classifier model: it can't write text." + case .unsupportedArchitecture(let type): + return "Architecture “\(type)” isn't supported by the installed mlx-lm." + } + } + } + + enum Verdict: Equatable, Sendable { + case compatible(modelType: String, isInstruct: Bool) + case incompatible(Reason) + } + + private static let tokenizerFiles = ["tokenizer.json", "tokenizer.model", "tokenizer_config.json"] + private static let speechNameFragments = ["tts", "whisper", "speech", "asr", "audio"] + private static let speechArchitectureFragments = ["Speech", "CTC", "Audio"] + + /// Checks run in this order: missing files, unreadable config, what kind of model it is, then + /// whether the installed mlx-lm knows the architecture. `supportedModelTypes` is nil when + /// mlx-lm is not installed yet; only the architecture check is skipped then (nothing can run + /// anyway, and hiding a downloaded model would read as "your download vanished"). + static func verdict( + repoName: String, + fileNames: [String], + config: [String: Any]?, + supportedModelTypes: Set? + ) -> Verdict { + guard fileNames.contains("config.json") else { return .incompatible(.missingConfig) } + guard fileNames.contains(where: { $0.hasSuffix(".safetensors") }) else { return .incompatible(.missingWeights) } + guard fileNames.contains(where: { tokenizerFiles.contains($0) }) else { return .incompatible(.missingTokenizer) } + guard let config, let modelType = config["model_type"] as? String else { + return .incompatible(.unreadableConfig) + } + + let lowerName = repoName.lowercased() + let architectures = config["architectures"] as? [String] ?? [] + func architecture(contains fragments: [String]) -> Bool { + architectures.contains { arch in fragments.contains { arch.contains($0) } } + } + if lowerName.contains("embed") || architecture(contains: ["Embedding"]) { + return .incompatible(.embeddingModel) + } + if lowerName.contains("rerank") { return .incompatible(.rerankerModel) } + if speechNameFragments.contains(where: { lowerName.contains($0) }) + || architecture(contains: speechArchitectureFragments) { + return .incompatible(.speechModel) + } + if architecture(contains: ["SequenceClassification"]) { return .incompatible(.classifierModel) } + + if let supportedModelTypes, !supportedModelTypes.contains(modelType) { + return .incompatible(.unsupportedArchitecture(modelType)) + } + return .compatible(modelType: modelType, isInstruct: isInstructName(repoName)) + } + + /// "-it", "instruct" or "chat" as a whole `-`/`_`/`.` separated token. + static func isInstructName(_ name: String) -> Bool { + let tokens = name.lowercased().split(whereSeparator: { "-_.".contains($0) }) + return tokens.contains { $0 == "it" || $0 == "instruct" || $0 == "chat" } + } +} diff --git a/Cotabby/Support/Runtime/MLX/MLXServerDesireResolver.swift b/Cotabby/Support/Runtime/MLX/MLXServerDesireResolver.swift new file mode 100644 index 000000000..8ce518e46 --- /dev/null +++ b/Cotabby/Support/Runtime/MLX/MLXServerDesireResolver.swift @@ -0,0 +1,75 @@ +import Foundation + +/// File overview: +/// The one rule that turns "what the user chose" plus "what this Mac has" into "what the MLX +/// server should be doing". `CotabbyAppEnvironment` re-evaluates it whenever any input publishes +/// and hands the answer to `MLXServerManager.apply(_:)`. Being pure, it is where every +/// combination is pinned by a test, instead of being scattered through Combine sinks. + +/// Exactly what to launch. Equality decides whether a running server must restart. +nonisolated struct MLXServerLaunch: Equatable, Sendable { + let executable: URL + let modelID: String + let promptCacheBytes: Int +} + +/// The answer the resolver gives: stop, explain why not, or run this exact launch. A plain value +/// with no behavior, so `MLXServerManager` can diff the previous and new answers and act only on a +/// change. Its lifetime is one resolver call; nothing stores it. +nonisolated enum MLXServerDesire: Equatable, Sendable { + /// MLX is not selected, or a prerequisite is still being checked or installed. + case stopped + /// MLX is selected but cannot run; the reason is shown to the user. + case unavailable(MLXUnavailableReason) + case running(MLXServerLaunch) +} + +/// The pure decision table behind "should the MLX server run, and with what". It owns no state and +/// has no collaborators: the caller gathers the inputs (engine setting, hardware, toolchain state, +/// selected model, cache scan) and passes them in. It is a caseless enum namespace rather than a +/// struct because there is nothing to construct, and it lives in `Support/` so every combination +/// stays testable without a process, a file system or Combine. +nonisolated enum MLXServerDesireResolver { + /// Every input of the decision, gathered by the caller into one value so the table reads as a + /// single function of its inputs. + struct Input: Equatable { + let engine: SuggestionEngineKind + let isAppleSilicon: Bool + let toolchain: MLXToolchainState + /// The selected model's repo id as the user stored it; validated here. + let modelID: String + /// Usable models from the cache scan; nil until the first scan finishes. + let cachedModels: [MLXCachedModel]? + let promptCacheBytes: Int + /// Downloaded models the compatibility rule turned away, with the reason. + var rejectedModels: [MLXRepoID: MLXModelCompatibility.Reason] = [:] + } + + static func desire(for input: Input) -> MLXServerDesire { + guard input.engine == .mlx else { return .stopped } + guard input.isAppleSilicon else { return .unavailable(.unsupportedHardware) } + switch input.toolchain { + case .checking, .installing: + // Transient: reporting "not installed" here would flash an attention dot at every launch. + return .stopped + case .notInstalled, .tooOld, .failed: + return .unavailable(.notInstalled) + case .ready(let toolchain): + guard let repoID = MLXRepoID(input.modelID) else { return .unavailable(.noModel) } + // The first cache scan has not finished: wait rather than claim the model is missing. + guard let cachedModels = input.cachedModels else { return .stopped } + guard cachedModels.contains(where: { $0.repoID == repoID }) else { + // Downloaded but turned away by the compatibility rule: say why, not "download it". + if let reason = input.rejectedModels[repoID] { + return .unavailable(.modelIncompatible(repoID, reason.message)) + } + return .unavailable(.modelNotDownloaded(repoID)) + } + return .running(MLXServerLaunch( + executable: toolchain.serverExecutable, + modelID: repoID.value, + promptCacheBytes: input.promptCacheBytes + )) + } + } +} diff --git a/Cotabby/Support/Runtime/MLX/MLXServerLogFilter.swift b/Cotabby/Support/Runtime/MLX/MLXServerLogFilter.swift new file mode 100644 index 000000000..3aeb92c7e --- /dev/null +++ b/Cotabby/Support/Runtime/MLX/MLXServerLogFilter.swift @@ -0,0 +1,81 @@ +import Foundation + +/// File overview: +/// Cleans `mlx_lm.server`'s stderr before it is quoted as a failure reason. The server is a Python +/// `http.server`, which prints one access line per request and a full traceback for every client +/// that disconnects early (Cotabby cancels a stream whenever the user types again). Left in, those +/// bury the one traceback that matters, such as `generation thread died`. Pure and stateless, so +/// it lives in `Support/` and is tested with a realistic transcript. +nonisolated enum MLXServerLogFilter { + /// The last `limit` lines of `lines` once access-log lines and disconnect tracebacks are gone. + static func relevantTail(_ lines: [String], limit: Int) -> [String] { + var kept: [String] = [] + var index = 0 + while index < lines.count { + let line = lines[index] + if isAccessLine(line) { + index += 1 + } else if line.hasPrefix("Traceback (most recent call last)") { + let end = tracebackEnd(in: lines, from: index) + let block = Array(lines[index...end]) + if isDisconnect(block) { + // http.server frames the traceback with a dashed rule and an "Exception occurred" + // line before it and a dashed rule after it. Drop those with the block. + while let last = kept.last, isDashRule(last) || last.hasPrefix("Exception occurred during processing") { + kept.removeLast() + } + index = end + 1 + if index < lines.count, isDashRule(lines[index]) { index += 1 } + } else { + kept.append(contentsOf: block) + index = end + 1 + } + } else { + kept.append(line) + index += 1 + } + } + return Array(kept.suffix(limit)) + } + + /// `127.0.0.1 - - [date] "POST /v1/completions HTTP/1.1" 200 -` + private static func isAccessLine(_ line: String) -> Bool { + line.range(of: #""[A-Z]+ \S+ HTTP/\d(\.\d)?" \d{3}"#, options: .regularExpression) != nil + } + + private static func isDashRule(_ line: String) -> Bool { + line.count >= 10 && line.allSatisfy { $0 == "-" } + } + + /// Index of the traceback's last line: its exception line, or further when the exception was + /// chained ("During handling of the above exception..."). A traceback still being written ends + /// at the last line available. + private static func tracebackEnd(in lines: [String], from start: Int) -> Int { + var index = start + 1 + while index < lines.count { + let line = lines[index] + let isFrame = line.hasPrefix(" ") || line.hasPrefix("\t") + if !isFrame { + // The exception line closes the block unless another traceback chains onto it. + let next = index + 1 < lines.count ? lines[index + 1] : nil + if let next, next.hasPrefix("During handling") || next.hasPrefix("The above exception") { + index += 2 + continue + } + if line.hasPrefix("During handling") || line.hasPrefix("The above exception") + || line.hasPrefix("Traceback (most recent call last)") { + index += 1 + continue + } + return index + } + index += 1 + } + return lines.count - 1 + } + + private static func isDisconnect(_ block: [String]) -> Bool { + block.contains { $0.hasPrefix("BrokenPipeError") || $0.hasPrefix("ConnectionResetError") + || $0.hasPrefix("ConnectionAbortedError") } + } +} diff --git a/Cotabby/Support/Runtime/MLX/MLXToolchainRules.swift b/Cotabby/Support/Runtime/MLX/MLXToolchainRules.swift new file mode 100644 index 000000000..dfc0c3498 --- /dev/null +++ b/Cotabby/Support/Runtime/MLX/MLXToolchainRules.swift @@ -0,0 +1,178 @@ +import Foundation + +/// File overview: +/// The pure rules behind finding, installing and running mlx-lm: where an install may live, how +/// to find the Python a launcher script runs, which versions are acceptable, and the exact command +/// lines and environments Cotabby uses. Services (`MLXToolchainProvisioner`, `MLXServerManager`, +/// `MLXModelDownloader`) call these instead of assembling strings themselves, so every flag and +/// environment variable is pinned by a unit test. +nonisolated enum MLXToolchainRules { + static let minimumVersion = "0.32.0" + static let pinnedVersion = "0.32.0" + + /// Prints the installed mlx-lm version from package metadata. Importing `mlx_lm` itself loads + /// MLX and its heavy dependencies (seconds), which made the probe slow enough to re-run only + /// at launch; reading `importlib.metadata` is milliseconds. Run with `python -I` so user + /// site-packages and PYTHON* variables cannot change what is found. + static let versionProbeScript = + "import sys, importlib.metadata as m; sys.stdout.write(m.version('mlx-lm'))" + + /// Prints a JSON list of every `model_type` the installed mlx-lm can load: its model modules + /// plus the remapped aliases (for example `mistral` loads through `llama`). + static let supportedModelTypesScript = """ + import json, pkgutil, sys + import mlx_lm.models as models + from mlx_lm.utils import MODEL_REMAPPING + names = [name for _, name, _ in pkgutil.iter_modules(models.__path__)] + sys.stdout.write(json.dumps(sorted(set(names) | set(MODEL_REMAPPING)))) + """ + + /// Downloads one repo into the Hugging Face cache. The repo id arrives as `sys.argv[1]`, never + /// interpolated into the code string. + static let downloadScript = + "import sys; from huggingface_hub import snapshot_download; snapshot_download(sys.argv[1])" + + /// GUI apps do not inherit the shell's PATH, so the usual install places are checked by path: + /// Cotabby's own install, uv's default tool bin, Homebrew. + static func serverCandidates(paths: MLXPaths, home: URL) -> [URL] { + [ + paths.ownServerExecutable, + home.appendingPathComponent(".local/bin/mlx_lm.server"), + URL(fileURLWithPath: "/opt/homebrew/bin/mlx_lm.server") + ] + } + + /// The Python a console-script launcher runs. Two shapes exist: + /// - `#!/abs/path/python` + /// - a `/bin/sh` trampoline (`'''exec' '/path with space/python' "$0" "$@"`) that uv and pip + /// write when the path has a space. + /// `#!/usr/bin/env python3` is rejected: a GUI app has no reliable PATH to resolve it with. + static func interpreter(fromScript text: String) -> URL? { + let lines = text.split(separator: "\n", maxSplits: 2, omittingEmptySubsequences: false).map(String.init) + guard let first = lines.first, first.hasPrefix("#!") else { return nil } + let shebang = first.dropFirst(2).trimmingCharacters(in: .whitespaces) + if shebang == "/bin/sh" { + let prefix = "'''exec' '" + guard lines.count > 1, lines[1].hasPrefix(prefix) else { return nil } + let rest = lines[1].dropFirst(prefix.count) + guard let end = rest.firstIndex(of: "'"), end > rest.startIndex else { return nil } + return URL(fileURLWithPath: String(rest[.. Bool { + guard let lhs = numericComponents(version), let rhs = numericComponents(minimum) else { return false } + for index in 0.. right } + } + return true + } + + /// The environment every MLX helper starts from: the app's environment without anything that + /// could make Python or uv load other code (PYTHONPATH, PYTHONHOME, an active virtualenv or + /// conda env, the user's own UV_* settings), with a fixed system PATH. + static func sanitizedEnvironment(_ base: [String: String]) -> [String: String] { + var environment = base.filter { key, _ in + !key.hasPrefix("PYTHON") && !key.hasPrefix("UV_") && !key.hasPrefix("CONDA_") + && key != "VIRTUAL_ENV" && key != "PIP_INDEX_URL" && key != "PIP_EXTRA_INDEX_URL" + } + environment["PATH"] = "/usr/bin:/bin:/usr/sbin:/sbin" + return environment + } + + static func installArguments() -> [String] { + ["tool", "install", "mlx-lm==\(pinnedVersion)", "--python", "3.12"] + } + + /// Everything uv writes stays under Cotabby's MLX folder; uv downloads its own Python and + /// never touches or uses the system one. + static func installEnvironment(paths: MLXPaths, base: [String: String]) -> [String: String] { + var environment = sanitizedEnvironment(base) + environment["UV_TOOL_DIR"] = paths.toolsDirectory.path + environment["UV_TOOL_BIN_DIR"] = paths.binDirectory.path + environment["UV_PYTHON_INSTALL_DIR"] = paths.pythonDirectory.path + environment["UV_CACHE_DIR"] = paths.cacheDirectory.path + environment["UV_NO_CONFIG"] = "1" + environment["UV_PYTHON_PREFERENCE"] = "only-managed" + // Compile bytecode at install time so the first server start does not pay for it. + environment["UV_COMPILE_BYTECODE"] = "1" + return environment + } + + /// The server binds loopback only, keeps two prompt caches (the current field and the one + /// before), and bounds their memory. + static func serverArguments(modelID: String, port: Int, promptCacheBytes: Int) -> [String] { + [ + "--model", modelID, + "--host", "127.0.0.1", + "--port", String(port), + "--prompt-cache-size", "2", + "--prompt-cache-bytes", String(promptCacheBytes), + "--log-level", "WARNING" + ] + } + + /// `HF_HUB_OFFLINE=1`: the server never downloads on its own; downloads happen only when the + /// user presses Download. + static func serverEnvironment(base: [String: String]) -> [String: String] { + var environment = sanitizedEnvironment(base) + environment["HF_HUB_OFFLINE"] = "1" + environment["HF_HUB_DISABLE_TELEMETRY"] = "1" + return environment + } + + static func downloadEnvironment(base: [String: String]) -> [String: String] { + var environment = sanitizedEnvironment(base) + environment.removeValue(forKey: "HF_HUB_OFFLINE") + environment["HF_HUB_DISABLE_TELEMETRY"] = "1" + // Progress bars would flood the captured stderr with carriage-return redraws. + environment["HF_HUB_DISABLE_PROGRESS_BARS"] = "1" + return environment + } + + /// Parses a `KERN_PROCARGS2` buffer: argc (Int32), the executable path, NUL padding, then argc + /// NUL-terminated arguments (the environment follows and is ignored). + static func parseProcArgs(_ bytes: [UInt8]) -> [String]? { + guard bytes.count > MemoryLayout.size else { return nil } + let argc = Int(bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self) }) + guard argc > 0 else { return nil } + var index = MemoryLayout.size + while index < bytes.count, bytes[index] != 0 { index += 1 } + while index < bytes.count, bytes[index] == 0 { index += 1 } + var arguments: [String] = [] + while arguments.count < argc, index < bytes.count { + let start = index + while index < bytes.count, bytes[index] != 0 { index += 1 } + // A non-UTF-8 argument cannot be one of the ASCII markers the caller looks for. + arguments.append(String(bytes: bytes[start.. Bool { + let runsServer = arguments.contains { ($0 as NSString).lastPathComponent == "mlx_lm.server" } + let loopback = zip(arguments, arguments.dropFirst()).contains { $0 == "--host" && $1 == "127.0.0.1" } + return runsServer && loopback + } + + private static func numericComponents(_ version: String) -> [Int]? { + let parts = version.trimmingCharacters(in: .whitespacesAndNewlines).split(separator: ".") + var numbers: [Int] = [] + for part in parts { + let digits = part.prefix { $0.isNumber } + guard let value = Int(digits) else { break } + numbers.append(value) + if digits.count != part.count { break } + } + return numbers.isEmpty ? nil : numbers + } +} diff --git a/Cotabby/Support/Settings/SettingsAttentionEvaluator.swift b/Cotabby/Support/Settings/SettingsAttentionEvaluator.swift index c919ee588..ca1eef5b2 100644 --- a/Cotabby/Support/Settings/SettingsAttentionEvaluator.swift +++ b/Cotabby/Support/Settings/SettingsAttentionEvaluator.swift @@ -21,6 +21,9 @@ enum SettingsAttentionEvaluator { let llamaRuntimeFailedReason: String? let endpointConfigurationError: String? let endpointConnectionFailedReason: String? + /// Why the MLX engine cannot generate (not installed, no model, failed), or nil when it can + /// or is merely starting. Defaulted so callers that predate MLX keep compiling. + var mlxProblem: String? } /// Returns the set of categories that should render an attention dot in the sidebar. @@ -44,6 +47,10 @@ enum SettingsAttentionEvaluator { if inputs.endpointConfigurationError != nil || inputs.endpointConnectionFailedReason != nil { categories.insert(.engineAndModel) } + case .mlx: + if inputs.mlxProblem != nil { + categories.insert(.engineAndModel) + } } return categories diff --git a/Cotabby/Support/Settings/SuggestionEngineSelectionPolicy.swift b/Cotabby/Support/Settings/SuggestionEngineSelectionPolicy.swift index dd85beac4..8f2bd494b 100644 --- a/Cotabby/Support/Settings/SuggestionEngineSelectionPolicy.swift +++ b/Cotabby/Support/Settings/SuggestionEngineSelectionPolicy.swift @@ -14,16 +14,19 @@ import Foundation /// why nothing is generated, and the choice comes back to life if Apple Intelligence becomes /// available later (for example after its model finishes downloading). enum SuggestionEngineSelectionPolicy { - /// Whether `engine` may be chosen right now. Only Apple Intelligence depends on runtime - /// availability; Open Source and the local endpoint are always selectable because their own - /// panes guide the user through downloading a model or configuring a server. + /// Whether `engine` may be chosen right now. Apple Intelligence depends on runtime availability, + /// and MLX on Apple Silicon. Open Source and the local endpoint are always selectable because + /// their own panes guide the user through downloading a model or configuring a server. static func isSelectable( _ engine: SuggestionEngineKind, - foundationModelAvailable: Bool + foundationModelAvailable: Bool, + isAppleSilicon: Bool = MLXHardware.isAppleSilicon ) -> Bool { switch engine { case .appleIntelligence: return foundationModelAvailable + case .mlx: + return isAppleSilicon case .llamaOpenSource, .openAICompatible: return true } @@ -33,9 +36,14 @@ enum SuggestionEngineSelectionPolicy { /// menu item still explains itself; the detailed reason stays in the pane callout. static func pickerLabel( for engine: SuggestionEngineKind, - foundationModelAvailable: Bool + foundationModelAvailable: Bool, + isAppleSilicon: Bool = MLXHardware.isAppleSilicon ) -> String { - guard isSelectable(engine, foundationModelAvailable: foundationModelAvailable) else { + guard isSelectable( + engine, + foundationModelAvailable: foundationModelAvailable, + isAppleSilicon: isAppleSilicon + ) else { return "\(engine.displayLabel) (Unavailable)" } return engine.displayLabel diff --git a/Cotabby/Support/Settings/SuggestionSettingsStore.swift b/Cotabby/Support/Settings/SuggestionSettingsStore.swift index 3bc74f9f2..21a8d4ad4 100644 --- a/Cotabby/Support/Settings/SuggestionSettingsStore.swift +++ b/Cotabby/Support/Settings/SuggestionSettingsStore.swift @@ -131,6 +131,8 @@ struct SuggestionSettingsStore { private static let openAICompatibleBaseURLDefaultsKey = "cotabbyOpenAICompatibleBaseURL" private static let openAICompatibleModelNameDefaultsKey = "cotabbyOpenAICompatibleModelName" private static let openAICompatibleAPIModeDefaultsKey = "cotabbyOpenAICompatibleAPIMode" + private static let mlxModelDefaultsKey = "cotabbyMLXModel" + private static let mlxPromptCacheBytesDefaultsKey = "cotabbyMLXPromptCacheBytes" private static let selectedWordCountPresetDefaultsKey = "cotabbySelectedWordCountPreset" private static let usingCustomWordCountRangeDefaultsKey = "cotabbyUsingCustomWordCountRange" private static let customWordCountLowWordsDefaultsKey = "cotabbyCustomWordCountLowWords" @@ -199,6 +201,17 @@ struct SuggestionSettingsStore { private static let pluggedInEngineDefaultsKey = "cotabbyPluggedInEngine" private static let pluggedInModelFilenameDefaultsKey = "cotabbyPluggedInModelFilename" private static let pluggedInEndpointModelNameDefaultsKey = "cotabbyPluggedInEndpointModelName" + private static let batteryMLXModelDefaultsKey = "cotabbyBatteryMLXModel" + private static let pluggedInMLXModelDefaultsKey = "cotabbyPluggedInMLXModel" + + /// 1 GiB holds the two prompt caches mlx_lm.server keeps for Cotabby's few-thousand-token + /// prompts, while stopping a long session from growing the server without bound. The clamp keeps + /// a hand-edited value between 256 MiB and 8 GiB. + static let defaultMLXPromptCacheBytes = 1 << 30 + + static func clampedMLXPromptCacheBytes(_ bytes: Int) -> Int { + min(max(bytes, 256 << 20), 8 << 30) + } /// Every persisted-preference key this store owns, in one place so `resetToDefaults` clears the /// full set. App state that is deliberately *not* a preference lives under other keys and is @@ -227,6 +240,8 @@ struct SuggestionSettingsStore { openAICompatibleBaseURLDefaultsKey, openAICompatibleModelNameDefaultsKey, openAICompatibleAPIModeDefaultsKey, + mlxModelDefaultsKey, + mlxPromptCacheBytesDefaultsKey, selectedWordCountPresetDefaultsKey, usingCustomWordCountRangeDefaultsKey, customWordCountLowWordsDefaultsKey, @@ -281,7 +296,9 @@ struct SuggestionSettingsStore { batteryEndpointModelNameDefaultsKey, pluggedInEngineDefaultsKey, pluggedInModelFilenameDefaultsKey, - pluggedInEndpointModelNameDefaultsKey + pluggedInEndpointModelNameDefaultsKey, + batteryMLXModelDefaultsKey, + pluggedInMLXModelDefaultsKey ] // MARK: - Load @@ -346,6 +363,10 @@ struct SuggestionSettingsStore { .string(forKey: Self.openAICompatibleAPIModeDefaultsKey) .flatMap(OpenAICompatibleAPIMode.init(rawValue:)) ?? .chatCompletions + let resolvedMLXModelID = userDefaults.string(forKey: Self.mlxModelDefaultsKey)? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + let resolvedMLXPromptCacheBytes = (userDefaults.object(forKey: Self.mlxPromptCacheBytesDefaultsKey) as? Int) + .map(Self.clampedMLXPromptCacheBytes) ?? Self.defaultMLXPromptCacheBytes let resolvedWordCountPreset: SuggestionWordCountPreset = { let storedRaw = userDefaults.string(forKey: Self.selectedWordCountPresetDefaultsKey) // Migrate the retired "3-7" raw value to its replacement "4-7" so users who picked @@ -574,6 +595,8 @@ struct SuggestionSettingsStore { let resolvedPluggedInModelFilename = userDefaults.string(forKey: Self.pluggedInModelFilenameDefaultsKey) ?? "" let resolvedPluggedInEndpointModelName = userDefaults .string(forKey: Self.pluggedInEndpointModelNameDefaultsKey) ?? "" + let resolvedBatteryMLXModelID = userDefaults.string(forKey: Self.batteryMLXModelDefaultsKey) ?? "" + let resolvedPluggedInMLXModelID = userDefaults.string(forKey: Self.pluggedInMLXModelDefaultsKey) ?? "" let data = SuggestionSettingsData( general: SuggestionGeneralSettings( @@ -597,7 +620,11 @@ struct SuggestionSettingsStore { pluggedInModelFilename: resolvedPluggedInModelFilename, pluggedInEndpointModelName: resolvedPluggedInEndpointModelName, isAppleLanguageFallbackEnabled: resolvedAppleLanguageFallbackEnabled, - keepsFallbackModelLoaded: resolvedKeepsFallbackModelLoaded + keepsFallbackModelLoaded: resolvedKeepsFallbackModelLoaded, + mlxModelID: resolvedMLXModelID, + mlxPromptCacheBytes: resolvedMLXPromptCacheBytes, + batteryMLXModelID: resolvedBatteryMLXModelID, + pluggedInMLXModelID: resolvedPluggedInMLXModelID ), completion: SuggestionCompletionSettings( selectedWordCountPreset: resolvedWordCountPreset, @@ -751,6 +778,10 @@ struct SuggestionSettingsStore { savePluggedInEngine(data.pluggedInEngine) savePluggedInModelFilename(data.pluggedInModelFilename) savePluggedInEndpointModelName(data.pluggedInEndpointModelName) + saveMLXModelID(data.mlxModelID) + saveMLXPromptCacheBytes(data.mlxPromptCacheBytes) + saveBatteryMLXModelID(data.batteryMLXModelID) + savePluggedInMLXModelID(data.pluggedInMLXModelID) // The custom indicator icon feature was removed; scrub any previously-persisted PNG so // users who picked one in an older build get the default cat icon back automatically. @@ -876,6 +907,22 @@ struct SuggestionSettingsStore { userDefaults.set(mode.rawValue, forKey: Self.openAICompatibleAPIModeDefaultsKey) } + func saveMLXModelID(_ modelID: String) { + userDefaults.set(modelID, forKey: Self.mlxModelDefaultsKey) + } + + func saveMLXPromptCacheBytes(_ bytes: Int) { + userDefaults.set(bytes, forKey: Self.mlxPromptCacheBytesDefaultsKey) + } + + func saveBatteryMLXModelID(_ modelID: String) { + userDefaults.set(modelID, forKey: Self.batteryMLXModelDefaultsKey) + } + + func savePluggedInMLXModelID(_ modelID: String) { + userDefaults.set(modelID, forKey: Self.pluggedInMLXModelDefaultsKey) + } + func saveAppleLanguageFallbackEnabled(_ enabled: Bool) { userDefaults.set(enabled, forKey: Self.appleLanguageFallbackEnabledDefaultsKey) } diff --git a/Cotabby/Support/Suggestion/Request/SuggestionRequestFactory.swift b/Cotabby/Support/Suggestion/Request/SuggestionRequestFactory.swift index e8711a205..4312dac7e 100644 --- a/Cotabby/Support/Suggestion/Request/SuggestionRequestFactory.swift +++ b/Cotabby/Support/Suggestion/Request/SuggestionRequestFactory.swift @@ -94,10 +94,10 @@ enum SuggestionRequestFactory { fieldPlaceholder: context.fieldPlaceholder ) : nil - // Typing history stays on this Mac. The provider already returns nothing for the endpoint - // engine; dropping it here as well keeps that guarantee in the one pure place every request - // passes through. - let activeHistoryExamples = settings.selectedEngine == .openAICompatible ? [] : historyExamples + // Typing history stays on this Mac. The provider already returns nothing for engines that + // are not on-device; dropping it here as well keeps that guarantee in the one pure place + // every request passes through. + let activeHistoryExamples = settings.selectedEngine.isOnDevice ? historyExamples : [] let maxPredictionTokens = activeMaxPredictionTokens( configuration: configuration, wordRange: settings.effectiveWordRange, @@ -116,7 +116,7 @@ enum SuggestionRequestFactory { // The endpoint backend shares this renderer, but adding document-tail content to a // network request needs its own disclosure and consent. Keep this new context local; // Apple's fallback request can use it because both eligible engines run on-device. - trailingText: settings.selectedEngine == .openAICompatible ? "" : context.trailingText, + trailingText: settings.selectedEngine.isOnDevice ? context.trailingText : "", maxSuffixCharacters: configuration.maxSuffixCharacters, customRules: customRules, extendedContext: activeExtendedContext, @@ -188,7 +188,8 @@ enum SuggestionRequestFactory { case .appleIntelligence: maxCharacters = configuration.maxPrefixCharactersFoundationModel maxWords = configuration.maxPrefixWordsFoundationModel - case .llamaOpenSource: + case .llamaOpenSource, .mlx: + // MLX serves the same base model family on the same prompt, so it shares the budget. maxCharacters = configuration.maxPrefixCharacters maxWords = configuration.maxPrefixWords case .openAICompatible: @@ -313,7 +314,7 @@ enum SuggestionRequestFactory { switch selectedEngine { case .appleIntelligence: return FoundationModelPromptRenderer.promptPreview(for: request) - case .llamaOpenSource: + case .llamaOpenSource, .mlx: return request.prompt case .openAICompatible: return request.prompt diff --git a/Cotabby/UI/MenuBar/MenuBarPresentationObserver.swift b/Cotabby/UI/MenuBar/MenuBarPresentationObserver.swift index 6f91d57a4..60e800fa9 100644 --- a/Cotabby/UI/MenuBar/MenuBarPresentationObserver.swift +++ b/Cotabby/UI/MenuBar/MenuBarPresentationObserver.swift @@ -8,15 +8,19 @@ import SwiftUI /// instead and calls `onPresent` whenever the panel becomes key or visible again. struct MenuBarPresentationObserver: NSViewRepresentable { let onPresent: () -> Void + /// Called when the panel is dismissed (resigned key, closed or fully occluded). + var onDismiss: () -> Void = {} func makeNSView(context: Context) -> MenuBarPresentationTrackingView { let view = MenuBarPresentationTrackingView() view.onPresent = onPresent + view.onDismiss = onDismiss return view } func updateNSView(_ nsView: MenuBarPresentationTrackingView, context: Context) { nsView.onPresent = onPresent + nsView.onDismiss = onDismiss } } @@ -27,6 +31,7 @@ struct MenuBarPresentationObserver: NSViewRepresentable { /// the menu should force one immediate permission read. final class MenuBarPresentationTrackingView: NSView { var onPresent: (() -> Void)? + var onDismiss: (() -> Void)? private weak var observedWindow: NSWindow? private var observers: [NSObjectProtocol] = [] @@ -75,7 +80,27 @@ final class MenuBarPresentationTrackingView: NSView { object: window, queue: .main ) { [weak self] _ in - self?.refreshIfVisible() + guard let self, let window = self.observedWindow else { return } + // Visible again re-presents; fully hidden is a dismissal. + if window.occlusionState.contains(.visible) { + self.refreshIfVisible() + } else { + self.onDismiss?() + } + }, + center.addObserver( + forName: NSWindow.didResignKeyNotification, + object: window, + queue: .main + ) { [weak self] _ in + self?.onDismiss?() + }, + center.addObserver( + forName: NSWindow.willCloseNotification, + object: window, + queue: .main + ) { [weak self] _ in + self?.onDismiss?() } ] } diff --git a/Cotabby/UI/MenuBar/MenuBarView.swift b/Cotabby/UI/MenuBar/MenuBarView.swift index 75564fa8d..32259909f 100644 --- a/Cotabby/UI/MenuBar/MenuBarView.swift +++ b/Cotabby/UI/MenuBar/MenuBarView.swift @@ -22,6 +22,8 @@ struct MenuBarView: View { @ObservedObject var suggestionSettings: SuggestionSettingsModel @ObservedObject var foundationModelAvailabilityService: FoundationModelAvailabilityService @ObservedObject var powerSourceMonitor: PowerSourceMonitor + @ObservedObject var mlxServerManager: MLXServerManager + @ObservedObject var menuResourceMonitor: MenuResourceMonitor let appUpdateManager: AppUpdateManager let onOpenSettings: () -> Void let onReportFeedback: () -> Void @@ -42,10 +44,15 @@ struct MenuBarView: View { .frame(width: 340) .modifier(MenuBarWindowBackgroundModifier()) .background( - MenuBarPresentationObserver { - permissionManager.refresh() - runtimeModel.refreshAvailableModels() - } + MenuBarPresentationObserver( + onPresent: { + permissionManager.refresh() + runtimeModel.refreshAvailableModels() + // The real window edge, not `onAppear`: the panel's content can outlive a close. + menuResourceMonitor.start() + }, + onDismiss: { menuResourceMonitor.stop() } + ) ) .background( MenuBarPopoverDismisserBinder( @@ -56,7 +63,10 @@ struct MenuBarView: View { .onAppear { permissionManager.refresh() runtimeModel.refreshAvailableModels() + menuResourceMonitor.start() } + // Sampling runs only while the popover is on screen. + .onDisappear { menuResourceMonitor.stop() } } // MARK: - Header @@ -199,6 +209,19 @@ struct MenuBarView: View { .foregroundStyle(.orange) } + // MLX pauses suggestions until its server is ready, so say why. + if suggestionSettings.selectedEngine == .mlx { + if let problem = mlxServerManager.state.problem { + Text(problem) + .font(.caption) + .foregroundStyle(.orange) + } else if case .starting = mlxServerManager.state { + Text(mlxServerManager.state.summary) + .font(.caption) + .foregroundStyle(.secondary) + } + } + if suggestionSettings.selectedEngine.supportsLocalModelManagement { modelRow } @@ -219,6 +242,16 @@ struct MenuBarView: View { .pickerStyle(.menu) } + // Live usage, refreshed only while this popover is visible (see the presentation + // observer above). + if let summary = menuResourceMonitor.summary { + Text(ResourceUsageSummary.menuLine(summary)) + .font(.caption) + .monospacedDigit() + .foregroundStyle(.secondary) + .lineLimit(1) + .minimumScaleFactor(0.85) + } } } .padding(.bottom, 12) @@ -385,6 +418,8 @@ struct MenuBarView: View { profile = .llama(filename: runtimeModel.selectedModelFilename ?? "") case .openAICompatible: profile = .openAICompatible(modelName: suggestionSettings.openAICompatibleModelName) + case .mlx: + profile = .mlx(modelID: suggestionSettings.mlxModelID) } applyProfileForCurrentPowerSource(profile) } diff --git a/Cotabby/UI/Onboarding/Welcome/WelcomeTemplateStepView.swift b/Cotabby/UI/Onboarding/Welcome/WelcomeTemplateStepView.swift index fd5c0a4cf..5338194ae 100644 --- a/Cotabby/UI/Onboarding/Welcome/WelcomeTemplateStepView.swift +++ b/Cotabby/UI/Onboarding/Welcome/WelcomeTemplateStepView.swift @@ -333,6 +333,8 @@ private struct TemplateCard: View { return "\(model.displayName) · \(model.approximateSizeLabel) download" case .openAICompatible: return "Local endpoint · configure later in Settings" + case .mlx: + return "MLX · set up in Settings" } } diff --git a/Cotabby/UI/Onboarding/Welcome/WelcomeView.swift b/Cotabby/UI/Onboarding/Welcome/WelcomeView.swift index 87638a488..a799ce200 100644 --- a/Cotabby/UI/Onboarding/Welcome/WelcomeView.swift +++ b/Cotabby/UI/Onboarding/Welcome/WelcomeView.swift @@ -531,7 +531,8 @@ extension WelcomeView { return false } return state == .downloaded || state.isDownloading - case .openAICompatible: + case .openAICompatible, .mlx: + // Onboarding never offers MLX, but a returning user may have it selected. return true } } diff --git a/Cotabby/UI/Settings/Panes/ContextPaneView.swift b/Cotabby/UI/Settings/Panes/ContextPaneView.swift index 5d6771752..c212e9bd5 100644 --- a/Cotabby/UI/Settings/Panes/ContextPaneView.swift +++ b/Cotabby/UI/Settings/Panes/ContextPaneView.swift @@ -82,10 +82,11 @@ struct ContextPaneView: View { /// The preview drives the real pipeline, so its privacy note must follow the selected engine. /// Apple Intelligence and Open Source keep the typed text on this Mac; a configured endpoint, /// which may be on the LAN or the internet, receives it exactly as it would from any field. + /// MLX keeps the text on this Mac too: its server listens only on 127.0.0.1. /// Cotabby can only speak for itself: what the endpoint retains is that server's policy. static func livePreviewPrivacyNote(for engine: SuggestionEngineKind) -> String { switch engine { - case .appleIntelligence, .llamaOpenSource: + case .appleIntelligence, .llamaOpenSource, .mlx: return "Nothing here is saved or shared; it only exercises the on-device model." case .openAICompatible: return "Cotabby doesn't save this text, but like any other field it's sent to your configured endpoint, " diff --git a/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Actions.swift b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Actions.swift index adbda697a..dd5047169 100644 --- a/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Actions.swift +++ b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Actions.swift @@ -20,6 +20,9 @@ extension EngineAndModelPaneView { case .llamaOpenSource: guard case .failed(let detail) = runtimeModel.state else { return nil } return SettingsPaneCallout(tone: .warning, message: detail) + case .mlx: + guard let problem = mlxServerManager.state.problem else { return nil } + return SettingsPaneCallout(tone: .warning, message: problem) case .openAICompatible: do { let configuration = try suggestionSettings.openAICompatibleConfiguration diff --git a/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+MLX.swift b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+MLX.swift new file mode 100644 index 000000000..27984c778 --- /dev/null +++ b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+MLX.swift @@ -0,0 +1,319 @@ +import SwiftUI + +/// MLX sections of the unified engine pane: the mlx-lm install, the model list (curated, cached +/// and custom), and the prompt-cache cap. The pane presents state and forwards actions. Installing, +/// running and downloading live in `MLXToolchainProvisioner`, `MLXServerManager` and +/// `MLXModelDownloader`, which `CotabbyAppEnvironment` owns. +extension EngineAndModelPaneView { + @ViewBuilder + var mlxSections: some View { + mlxRuntimeSection + mlxModelSection + mlxAdvancedSection + } + + // MARK: - Runtime + + private var mlxRuntimeSection: some View { + Section("MLX") { + LabeledContent { + mlxRuntimeControls + } label: { + SettingsRowLabel(title: "MLX", description: mlxRuntimeDescription, systemImage: "bolt.fill") + } + .settingsItem(.mlxRuntime) + + if let tail = mlxVisibleLogTail, !tail.isEmpty { + Text(tail.joined(separator: "\n")) + .font(.caption.monospaced()) + .foregroundStyle(.secondary) + .textSelection(.enabled) + .lineLimit(20) + } + } + .alert("Remove MLX?", isPresented: $showsMLXRemoveConfirmation) { + Button("Remove", role: .destructive) { + Task { + mlxIsBusy = true + defer { mlxIsBusy = false } + await mlxToolchainProvisioner.remove() + } + } + Button("Cancel", role: .cancel) {} + } message: { + Text("Deletes Cotabby's copy of mlx-lm and its Python. Downloaded models stay in the Hugging Face cache.") + } + } + + private var mlxRuntimeDescription: String { + switch mlxToolchainProvisioner.state { + case .checking: + return "Checking for MLX…" + case .notInstalled: + return "Cotabby installs Apple's mlx-lm and runs it on this Mac (about 250 MB, plus a model)." + case .tooOld(let version, let path): + return "Found mlx-lm \(version) at \(path). Cotabby needs \(MLXToolchainRules.minimumVersion) " + + "or newer; Install adds Cotabby's own copy." + case .installing(let phase): + return phase.label + case .ready(let toolchain): + let owner = toolchain.isCotabbyInstall ? "Cotabby's" : "Your" + return "\(owner) mlx-lm \(toolchain.version). \(mlxServerManager.state.summary)" + case .failed(let reason, _): + return reason + } + } + + @ViewBuilder + private var mlxRuntimeControls: some View { + switch mlxToolchainProvisioner.state { + case .checking: + ProgressView().controlSize(.small) + case .notInstalled, .tooOld, .failed: + Button("Install MLX") { mlxToolchainProvisioner.install() } + .buttonStyle(.borderedProminent) + case .installing: + HStack(spacing: 8) { + ProgressView().controlSize(.small) + Button("Cancel") { mlxToolchainProvisioner.cancelInstall() } + } + case .ready(let toolchain): + HStack(spacing: 8) { + if case .failed = mlxServerManager.state { + Button("Retry") { + Task { + await mlxToolchainProvisioner.rescan(models: mlxModelDownloader) + await mlxServerManager.retry() + } + } + } + if toolchain.isCotabbyInstall { + Button("Remove MLX…", role: .destructive) { showsMLXRemoveConfirmation = true } + .disabled(mlxIsBusy) + } + } + } + } + + /// The server's failure tail, else the install's. + private var mlxVisibleLogTail: [String]? { + if case .failed(_, let tail) = mlxServerManager.state { return tail } + if case .failed(_, let tail) = mlxToolchainProvisioner.state { return tail } + return nil + } + + // MARK: - Models + + private var mlxModelSection: some View { + Section("Model") { + let options = MLXModelCatalog.options(cached: mlxModelDownloader.cachedModels ?? []) + ForEach(options) { option in + // `settingsItem` sets the row's `.id`; only one row may carry the search anchor. + if option.id == options.first?.id { + mlxModelRow(option).settingsItem(.mlxModel) + } else { + mlxModelRow(option) + } + } + mlxCustomRepoRow(downloadWithoutRow: MLXModelCatalog.downloadWithoutRow( + mlxModelDownloader.download, options: options)) + } + .alert( + "Delete MLX Model?", + isPresented: Binding( + get: { pendingMLXDeletion != nil }, + set: { if !$0 { pendingMLXDeletion = nil } } + ), + presenting: pendingMLXDeletion + ) { option in + Button("Delete", role: .destructive) { deleteMLXModel(option) } + Button("Cancel", role: .cancel) {} + } message: { option in + Text("Remove \(option.repoID.value) from the shared Hugging Face cache? Other apps that use this cache lose it too.") + } + } + + private func mlxModelRow(_ option: MLXModelOption) -> some View { + let download = mlxModelDownloader.download?.repoID == option.repoID ? mlxModelDownloader.download : nil + return HStack(alignment: .firstTextBaseline, spacing: 8) { + VStack(alignment: .leading, spacing: 3) { + HStack(spacing: 6) { + Text(option.displayName) + if option.isRecommended { + Text("Recommended").font(.caption).foregroundStyle(.secondary) + } + } + Text(([ByteCountFormatter.string(fromByteCount: option.sizeBytes, countStyle: .file)] + [option.note].compactMap { $0 }) + .joined(separator: " · ")) + .font(.caption) + .foregroundStyle(.secondary) + if let download { mlxDownloadStatus(download) } + } + Spacer(minLength: 8) + mlxModelActions(option, isDownloading: download?.isActive == true) + } + } + + /// Progress while a download runs, and its error once it failed. + @ViewBuilder + private func mlxDownloadStatus(_ download: MLXDownloadState) -> some View { + if download.isActive { + if let fraction = download.fraction { + ProgressView(value: fraction) + } else { + ProgressView().progressViewStyle(.linear) + } + } + if let failure = download.failure { + Text(failure).font(.caption).foregroundStyle(.red) + } + } + + @ViewBuilder + private func mlxModelActions(_ option: MLXModelOption, isDownloading: Bool) -> some View { + if option.isDownloaded { + if suggestionSettings.mlxModelID == option.repoID.value { + Label("In Use", systemImage: "checkmark.circle.fill") + .foregroundStyle(.green) + } else { + Button("Use") { suggestionSettings.setMLXModelID(option.repoID.value) } + .disabled(mlxIsBusy) + } + Button { + pendingMLXDeletion = option + } label: { + Image(systemName: "trash") + } + .buttonStyle(.borderless) + .disabled(mlxIsBusy) + .help("Delete from the Hugging Face cache") + } else if isDownloading { + Button("Cancel") { mlxModelDownloader.cancelDownload() } + } else { + Button("Download") { startMLXDownload(option.repoID) } + .disabled(!canStartMLXDownload) + .help(mlxToolchainProvisioner.state.toolchain == nil ? "Install MLX first." : "") + } + } + + /// `downloadWithoutRow` is a custom model's download, which has no catalog row until it is + /// cached; its progress, Cancel and failure are shown here instead. + private func mlxCustomRepoRow(downloadWithoutRow: MLXDownloadState?) -> some View { + VStack(alignment: .leading, spacing: 6) { + SettingsRowLabel( + title: "Other MLX Model", + description: "A Hugging Face model id or link, for example mlx-community/Qwen3-4B-4bit. " + + "Base models complete text better than chat models.", + systemImage: "plus.circle" + ) + HStack(spacing: 8) { + TextField("", text: $mlxCustomRepoDraft, prompt: Text("owner/model")) + .labelsHidden() + .textFieldStyle(.roundedBorder) + .onSubmit(checkMLXCustomRepo) + .onChange(of: mlxCustomRepoDraft) { _, _ in mlxCustomValidation = nil } + Button("Check", action: checkMLXCustomRepo) + .disabled(mlxCustomRepoDraft.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + || mlxIsCheckingCustomRepo || mlxIsBusy) + if case .valid(let repoID, _, _) = mlxCustomValidation { + Button("Download") { startMLXDownload(repoID) } + .buttonStyle(.borderedProminent) + .disabled(!canStartMLXDownload) + } + } + if case .valid(let repoID, let bytes, let isInstruct) = mlxCustomValidation { + Text("\(repoID.value) · \(ByteCountFormatter.string(fromByteCount: bytes, countStyle: .file))") + .font(.caption) + .foregroundStyle(.secondary) + if isInstruct { + Text(MLXModelCatalog.instructNote).font(.caption).foregroundStyle(.secondary) + } + } else if let message = mlxCustomValidation?.message { + Text(message).font(.caption).foregroundStyle(.orange) + } + if let download = downloadWithoutRow { + HStack(alignment: .firstTextBaseline, spacing: 8) { + VStack(alignment: .leading, spacing: 3) { + Text(download.repoID.value).font(.caption) + mlxDownloadStatus(download) + } + Spacer(minLength: 8) + if download.isActive { + Button("Cancel") { mlxModelDownloader.cancelDownload() } + } + } + } + } + .settingsItem(.mlxCustomModel) + } + + // MARK: - Advanced + + private var mlxAdvancedSection: some View { + Section { + DisclosureGroup("Advanced") { + Picker(selection: Binding( + get: { suggestionSettings.mlxPromptCacheBytes }, + set: { suggestionSettings.setMLXPromptCacheBytes($0) } + )) { + ForEach(promptCacheOptions, id: \.self) { bytes in + Text(ByteCountFormatter.string(fromByteCount: Int64(bytes), countStyle: .memory)).tag(bytes) + } + } label: { + SettingsRowLabel( + title: "Prompt Cache Memory", + description: "The most memory MLX keeps for reusing recent prompts. More can speed up " + + "suggestions in long documents; changing it restarts MLX.", + systemImage: "memorychip" + ) + } + .pickerStyle(.menu) + .settingsItem(.mlxPromptCache) + } + } + } + + /// The four presets, plus the stored value when it is off-preset (the store clamps to + /// 256 MiB...8 GiB), so the menu always has a tag for the current selection. + private var promptCacheOptions: [Int] { + let presets = [512 << 20, 1 << 30, 2 << 30, 4 << 30] + let current = suggestionSettings.mlxPromptCacheBytes + return presets.contains(current) ? presets : (presets + [current]).sorted() + } + + // MARK: - Actions + + private var canStartMLXDownload: Bool { + mlxToolchainProvisioner.state.toolchain != nil && mlxModelDownloader.download?.isActive != true + && !mlxIsBusy + } + + func startMLXDownload(_ repoID: MLXRepoID) { + guard let toolchain = mlxToolchainProvisioner.state.toolchain else { return } + mlxModelDownloader.startDownload(repoID, toolchain: toolchain) + } + + func checkMLXCustomRepo() { + let draft = mlxCustomRepoDraft + mlxIsCheckingCustomRepo = true + Task { + let result = await mlxModelDownloader.validateCustomRepo(draft, toolchain: mlxToolchainProvisioner.state.toolchain) + mlxIsCheckingCustomRepo = false + // The field may have changed while the check ran; a stale answer would mislabel it. + guard draft == mlxCustomRepoDraft else { return } + mlxCustomValidation = result + } + } + + /// Deletes the model from the cache and does not stop the server itself. The pipeline in + /// `CotabbyAppEnvironment` stops it when the cache rescan publishes without the model, which + /// keeps stops on the serialized, de-duplicated desire path. A running server keeps its + /// memory-mapped weights valid after the files are unlinked, so deleting first is safe. + func deleteMLXModel(_ option: MLXModelOption) { + Task { + mlxIsBusy = true + defer { mlxIsBusy = false } + await mlxModelDownloader.delete(option.repoID, toolchain: mlxToolchainProvisioner.state.toolchain) + } + } +} diff --git a/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Power.swift b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Power.swift index 1ce2b3dd6..96ad65173 100644 --- a/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Power.swift +++ b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView+Power.swift @@ -80,6 +80,10 @@ extension EngineAndModelPaneView { ForEach(endpointPowerModels, id: \.self) { modelName in Text("Endpoint · \(modelName)").tag(PowerProfile.openAICompatible(modelName: modelName)) } + + ForEach(mlxPowerModels, id: \.self) { modelID in + Text("MLX · \(modelID)").tag(PowerProfile.mlx(modelID: modelID)) + } } label: { SettingsRowLabel( title: title, @@ -113,6 +117,9 @@ extension EngineAndModelPaneView { if case .openAICompatible(let modelName) = profile, modelName.isEmpty { return .openAICompatible(modelName: suggestionSettings.openAICompatibleModelName) } + if case .mlx(let modelID) = profile, modelID.isEmpty { + return .mlx(modelID: suggestionSettings.mlxModelID) + } return profile } @@ -125,4 +132,15 @@ extension EngineAndModelPaneView { } return names } + + /// Downloaded MLX models plus the configured one, offered only where MLX can run. + var mlxPowerModels: [String] { + guard MLXHardware.isAppleSilicon else { return [] } + var ids = (mlxModelDownloader.cachedModels ?? []).map(\.repoID.value) + let configured = suggestionSettings.mlxModelID + if !configured.isEmpty, !ids.contains(configured) { + ids.append(configured) + } + return ids + } } diff --git a/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView.swift b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView.swift index e937af5e5..bcd64b358 100644 --- a/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView.swift +++ b/Cotabby/UI/Settings/Panes/Engine/EngineAndModelPaneView.swift @@ -9,6 +9,9 @@ import SwiftUI struct EngineAndModelPaneView: View { @ObservedObject var suggestionSettings: SuggestionSettingsModel @ObservedObject var openAICompatibleConnectionModel: OpenAICompatibleConnectionModel + @ObservedObject var mlxToolchainProvisioner: MLXToolchainProvisioner + @ObservedObject var mlxServerManager: MLXServerManager + @ObservedObject var mlxModelDownloader: MLXModelDownloader @ObservedObject var foundationModelAvailabilityService: FoundationModelAvailabilityService @ObservedObject var runtimeModel: RuntimeBootstrapModel @ObservedObject var modelDownloadManager: ModelDownloadManager @@ -24,6 +27,13 @@ struct EngineAndModelPaneView: View { /// reads, so the toggle and the model scan stay in sync. LM Studio models are an additive, /// read-only source; Cotabby's own folder is always scanned and is always the download target. @AppStorage(BundledRuntimeLocator.lmStudioSourceEnabledKey) var lmStudioSourceEnabled = false + @State var mlxCustomRepoDraft = "" + @State var mlxCustomValidation: MLXRepoValidation? + @State var mlxIsCheckingCustomRepo = false + @State var pendingMLXDeletion: MLXModelOption? + @State var showsMLXRemoveConfirmation = false + /// True while a model delete or an MLX remove runs; the taps that would conflict with it are disabled. + @State var mlxIsBusy = false var body: some View { SettingsPaneScaffold(callout: callout) { @@ -47,6 +57,7 @@ struct EngineAndModelPaneView: View { title: "Engine", description: "Apple Intelligence runs on-device using macOS's built-in model " + "(newer Apple Silicon Macs only). Open Source runs a downloaded model file. " + + "MLX runs an MLX model with Apple's mlx-lm, which Cotabby installs and manages. " + "Local Endpoint connects to an OpenAI-compatible server you manage.", systemImage: "cpu" ) @@ -64,15 +75,23 @@ struct EngineAndModelPaneView: View { openSourceSections case .openAICompatible: openAICompatibleSections + case .mlx: + mlxSections } } .onAppear { foundationModelAvailabilityService.refresh() + // Detection is otherwise launch-only; another tool may have installed or removed + // mlx-lm or a model since. + if suggestionSettings.selectedEngine == .mlx { + Task { await mlxToolchainProvisioner.rescan(models: mlxModelDownloader) } + } suggestionSettings.initializePowerProfiles( currentEngine: suggestionSettings.selectedEngine, currentModelFilename: runtimeModel.selectedModelFilename, - currentEndpointModelName: suggestionSettings.openAICompatibleModelName + currentEndpointModelName: suggestionSettings.openAICompatibleModelName, + currentMLXModelID: suggestionSettings.mlxModelID ) lmStudioModelsURL = BundledRuntimeLocator.lmStudioModelsDirectoryIfAvailable() diff --git a/Cotabby/UI/Settings/Panes/HomePaneView.swift b/Cotabby/UI/Settings/Panes/HomePaneView.swift index cd15f4470..7ba628116 100644 --- a/Cotabby/UI/Settings/Panes/HomePaneView.swift +++ b/Cotabby/UI/Settings/Panes/HomePaneView.swift @@ -18,6 +18,7 @@ struct HomePaneView: View { @ObservedObject var foundationModelAvailabilityService: FoundationModelAvailabilityService @ObservedObject var runtimeModel: RuntimeBootstrapModel @ObservedObject var openAICompatibleConnectionModel: OpenAICompatibleConnectionModel + @ObservedObject var mlxServerManager: MLXServerManager let attentionCategories: Set @State private var query = "" @@ -322,6 +323,9 @@ struct HomePaneView: View { switch suggestionSettings.selectedEngine { case .appleIntelligence: return foundationModelAvailabilityService.isAvailable ? "Ready on this Mac" : "Unavailable" + case .mlx: + if let problem = mlxServerManager.state.problem { return problem } + return suggestionSettings.mlxModelID.isEmpty ? "No model selected" : suggestionSettings.mlxModelID case .llamaOpenSource: let selected = runtimeModel.availableModels .first { $0.filename == runtimeModel.selectedModelFilename } diff --git a/Cotabby/UI/Settings/Panes/PerformancePaneView.swift b/Cotabby/UI/Settings/Panes/PerformancePaneView.swift index ac0c7a93f..0e1790af9 100644 --- a/Cotabby/UI/Settings/Panes/PerformancePaneView.swift +++ b/Cotabby/UI/Settings/Panes/PerformancePaneView.swift @@ -167,7 +167,16 @@ struct PerformancePaneView: View { .foregroundStyle(.secondary) } } header: { - Text("Live Resource Usage") + HStack(spacing: 6) { + Text("Live Resource Usage") + // CPU and Memory add the MLX server process when MLX is running it, so the numbers + // are not mistaken for Cotabby's own. + if systemMetricsStore.samples.last?.includesMLXServer == true { + Text("CPU and Memory include the MLX server") + .font(.caption) + .foregroundStyle(.secondary) + } + } } footer: { Text( "Updated every second while this pane is open. CPU can exceed 100% across multiple " + diff --git a/Cotabby/UI/Settings/SettingsContainerView.swift b/Cotabby/UI/Settings/SettingsContainerView.swift index 89726b44e..818af1c4e 100644 --- a/Cotabby/UI/Settings/SettingsContainerView.swift +++ b/Cotabby/UI/Settings/SettingsContainerView.swift @@ -18,6 +18,9 @@ struct SettingsContainerView: View { let permissionGuidanceController: PermissionGuidanceController @ObservedObject var suggestionSettings: SuggestionSettingsModel @ObservedObject var openAICompatibleConnectionModel: OpenAICompatibleConnectionModel + @ObservedObject var mlxToolchainProvisioner: MLXToolchainProvisioner + @ObservedObject var mlxServerManager: MLXServerManager + @ObservedObject var mlxModelDownloader: MLXModelDownloader @ObservedObject var foundationModelAvailabilityService: FoundationModelAvailabilityService @ObservedObject var runtimeModel: RuntimeBootstrapModel @ObservedObject var modelDownloadManager: ModelDownloadManager @@ -102,7 +105,8 @@ struct SettingsContainerView: View { foundationModelAvailable: foundationModelAvailabilityService.isAvailable, llamaRuntimeFailedReason: runtimeModel.state.failureDetail, endpointConfigurationError: endpointConfigurationError, - endpointConnectionFailedReason: openAICompatibleConnectionModel.state.failureDetail + endpointConnectionFailedReason: openAICompatibleConnectionModel.state.failureDetail, + mlxProblem: mlxServerManager.state.problem ) ) } @@ -118,6 +122,7 @@ struct SettingsContainerView: View { foundationModelAvailabilityService: foundationModelAvailabilityService, runtimeModel: runtimeModel, openAICompatibleConnectionModel: openAICompatibleConnectionModel, + mlxServerManager: mlxServerManager, attentionCategories: attentionCategories ) case .general: @@ -137,6 +142,9 @@ struct SettingsContainerView: View { EngineAndModelPaneView( suggestionSettings: suggestionSettings, openAICompatibleConnectionModel: openAICompatibleConnectionModel, + mlxToolchainProvisioner: mlxToolchainProvisioner, + mlxServerManager: mlxServerManager, + mlxModelDownloader: mlxModelDownloader, foundationModelAvailabilityService: foundationModelAvailabilityService, runtimeModel: runtimeModel, modelDownloadManager: modelDownloadManager, diff --git a/Cotabby/UI/Settings/SettingsIndex.swift b/Cotabby/UI/Settings/SettingsIndex.swift index b4e4fdfae..453d91f25 100644 --- a/Cotabby/UI/Settings/SettingsIndex.swift +++ b/Cotabby/UI/Settings/SettingsIndex.swift @@ -82,6 +82,10 @@ enum SettingsItem: String, CaseIterable, Identifiable { case endpointAPIKey case endpointStatus case endpointModel + case mlxRuntime + case mlxModel + case mlxCustomModel + case mlxPromptCache // Shortcuts case acceptanceMode case acceptWord @@ -175,6 +179,10 @@ enum SettingsItem: String, CaseIterable, Identifiable { case .endpointAPIKey: return "Endpoint API Key" case .endpointStatus: return "Endpoint Status" case .endpointModel: return "Endpoint Model" + case .mlxRuntime: return "MLX" + case .mlxModel: return "MLX Model" + case .mlxCustomModel: return "Other MLX Model" + case .mlxPromptCache: return "MLX Prompt Cache Memory" case .acceptanceMode: return "Acceptance Mode" case .acceptWord: return "Accept Word" case .acceptEntireSuggestion: return "Accept Entire Suggestion" @@ -263,6 +271,10 @@ enum SettingsItem: String, CaseIterable, Identifiable { case .endpointAPIKey: return "key" case .endpointStatus: return "info.circle" case .endpointModel: return "shippingbox" + case .mlxRuntime: return "bolt.fill" + case .mlxModel: return "shippingbox" + case .mlxCustomModel: return "plus.circle" + case .mlxPromptCache: return "memorychip" case .acceptanceMode: return "textformat.abc" case .acceptWord: return "arrow.right.to.line" case .acceptEntireSuggestion: return "text.insert" @@ -309,7 +321,8 @@ enum SettingsItem: String, CaseIterable, Identifiable { .appleLanguageFallbackModel, .modelStatus, .selectedModel, .lowPowerModeAutoDisable, .powerBasedModelSwitching, .batteryModel, .pluggedInModel, .downloadModels, .huggingFaceBrowser, .modelsFolder, .lmStudio, - .endpointBaseURL, .endpointAPIMode, .endpointAPIKey, .endpointStatus, .endpointModel: + .endpointBaseURL, .endpointAPIMode, .endpointAPIKey, .endpointStatus, .endpointModel, + .mlxRuntime, .mlxModel, .mlxCustomModel, .mlxPromptCache: return .engineAndModel case .acceptanceMode, .acceptWord, .acceptEntireSuggestion, .toggleTabby: return .shortcuts @@ -395,6 +408,10 @@ enum SettingsItem: String, CaseIterable, Identifiable { case .endpointAPIKey: return "Optional bearer token stored securely in Keychain." case .endpointStatus: return "Whether Cotabby can reach the server and list its models." case .endpointModel: return "The model identifier sent to the configured endpoint." + case .mlxRuntime: return "Install, remove or restart Apple's mlx-lm, which runs MLX models." + case .mlxModel: return "Download and choose the MLX model used for suggestions." + case .mlxCustomModel: return "Use any MLX model from Hugging Face by its id." + case .mlxPromptCache: return "How much memory MLX may use to reuse recent prompts." case .acceptanceMode: return "Whether the accept key takes a word or a phrase." case .acceptWord: return "The key that inserts the next word." case .acceptEntireSuggestion: return "The key, or the Accept Word key pressed twice, that inserts the whole suggestion." @@ -609,6 +626,14 @@ enum SettingsItem: String, CaseIterable, Identifiable { return ["connect", "connection", "server", "status", "refresh", "health", "models"] case .endpointModel: return ["model", "identifier", "ollama", "lm studio", "vllm", "endpoint"] + case .mlxRuntime: + return ["mlx", "mlx-lm", "apple silicon", "install", "python", "server", "engine"] + case .mlxModel: + return ["mlx", "model", "download", "gemma", "hugging face", "cache"] + case .mlxCustomModel: + return ["mlx", "custom", "hugging face", "repo", "model id", "mlx-community"] + case .mlxPromptCache: + return ["mlx", "prompt cache", "memory", "kv cache", "ram"] case .acceptanceMode: return ["acceptance", "word", "phrase", "mode", "tap", "hold", "behavior", "how to accept"] diff --git a/CotabbyTests/App/Coordinators/Suggestion/SuggestionCoordinatorTypingPredictionTests.swift b/CotabbyTests/App/Coordinators/Suggestion/SuggestionCoordinatorTypingPredictionTests.swift index f5e4f63c7..e8a3a8f4e 100644 --- a/CotabbyTests/App/Coordinators/Suggestion/SuggestionCoordinatorTypingPredictionTests.swift +++ b/CotabbyTests/App/Coordinators/Suggestion/SuggestionCoordinatorTypingPredictionTests.swift @@ -330,6 +330,19 @@ final class SuggestionCoordinatorTypingPredictionTests: XCTestCase { XCTAssertNotEqual(rig.coordinator.currentWorkID, workID) } + func testMLXDoesNotEnableBackgroundCollectionOrRetainRequestsAcrossKeys() async { + let engine = ControlledTypingEngine() + let rig = makeRig(engine, engineKind: .mlx) + defer { rig.coordinator.stop(); engine.finishAll() } + rig.coordinator.schedulePrediction() + await waitUntil { engine.requests.count == 1 } + XCTAssertNil(rig.coordinator.typingPrediction) + XCTAssertFalse(engine.hasPartialCallback(request: 0)) + let workID = rig.coordinator.currentWorkID + type("you ", in: rig) + XCTAssertNotEqual(rig.coordinator.currentWorkID, workID) + } + func testDisabledPredictionRestartsOnTypingWithoutChangingStreamingPreference() async { for streaming in [false, true] { let engine = ControlledTypingEngine() diff --git a/CotabbyTests/Models/Context/PermissionAndContextModelTests.swift b/CotabbyTests/Models/Context/PermissionAndContextModelTests.swift index 8438ca161..633ddff39 100644 --- a/CotabbyTests/Models/Context/PermissionAndContextModelTests.swift +++ b/CotabbyTests/Models/Context/PermissionAndContextModelTests.swift @@ -96,5 +96,6 @@ final class VisualContextModelTests: XCTestCase { XCTAssertEqual(VisualContextConfiguration.forEngine(.openAICompatible), .default) XCTAssertEqual(VisualContextConfiguration.forEngine(.appleIntelligence), .local) XCTAssertEqual(VisualContextConfiguration.forEngine(.llamaOpenSource), .local) + XCTAssertEqual(VisualContextConfiguration.forEngine(.mlx), .local) } } diff --git a/CotabbyTests/Models/Runtime/MLXModelsTests.swift b/CotabbyTests/Models/Runtime/MLXModelsTests.swift new file mode 100644 index 000000000..14a6cefb8 --- /dev/null +++ b/CotabbyTests/Models/Runtime/MLXModelsTests.swift @@ -0,0 +1,115 @@ +import XCTest +@testable import Cotabby + +/// File overview: +/// Tests for the MLX value types in `Models/Runtime/MLXModels.swift`: repo id validation (the +/// security boundary before a model id reaches a command line), the curated catalog merge, the +/// server state's user-facing messages, and the install paths. All of them are pure, so no test +/// here touches the network, the file system outside a fixed URL, or a process. + +final class MLXRepoIDTests: XCTestCase { + func test_acceptsOwnerSlashNameAndTrims() { + XCTAssertEqual(MLXRepoID(" mlx-community/gemma-4-e4b-5bit \n")?.value, "mlx-community/gemma-4-e4b-5bit") + XCTAssertEqual(MLXRepoID("Org_1/Model.v2")?.value, "Org_1/Model.v2") + } + + func test_reducesAPastedHuggingFaceURL() { + XCTAssertEqual(MLXRepoID("https://huggingface.co/mlx-community/gemma-4-e4b-5bit")?.value, + "mlx-community/gemma-4-e4b-5bit") + XCTAssertEqual(MLXRepoID("https://huggingface.co/mlx-community/gemma-4-e4b-5bit/tree/main")?.value, + "mlx-community/gemma-4-e4b-5bit") + XCTAssertNil(MLXRepoID("https://example.com/mlx-community/x")) + } + + /// Dataset and Space links must not turn into a wrong model id ("datasets/org"), and a link + /// copied without its scheme is still recognized. + func test_rejectsNonModelURLsAndAcceptsALinkWithoutScheme() { + XCTAssertNil(MLXRepoID("https://huggingface.co/datasets/org/name")) + XCTAssertNil(MLXRepoID("https://huggingface.co/spaces/org/app")) + XCTAssertNil(MLXRepoID("https://huggingface.co/models")) + XCTAssertEqual(MLXRepoID("huggingface.co/mlx-community/gemma-4-e4b-5bit")?.value, + "mlx-community/gemma-4-e4b-5bit") + XCTAssertEqual(MLXRepoID("www.huggingface.co/org/name/tree/main")?.value, "org/name") + } + + func test_rejectsAnythingElse() { + for raw in ["", "gemma", "org/name/extra", "../../etc", "org/..", "org/a b", "org/-x", + "org/.hidden", "org/a--b", "org/a;rm", "/name", "org/"] { + XCTAssertNil(MLXRepoID(raw), raw) + } + } + + func test_cacheFolderNameRoundTrips() { + let id = MLXRepoID("mlx-community/gemma-4-e4b-5bit")! + XCTAssertEqual(id.cacheFolderName, "models--mlx-community--gemma-4-e4b-5bit") + XCTAssertEqual(MLXRepoID(cacheFolderName: "models--mlx-community--gemma-4-e4b-5bit"), id) + XCTAssertNil(MLXRepoID(cacheFolderName: "datasets--a--b")) + XCTAssertNil(MLXRepoID(cacheFolderName: "models--onlyone")) + } +} + +final class MLXModelCatalogTests: XCTestCase { + private func cached(_ id: String, size: Int64 = 1, instruct: Bool = false) -> MLXCachedModel { + MLXCachedModel(repoID: MLXRepoID(id)!, snapshotURL: URL(fileURLWithPath: "/tmp/\(id)"), + sizeBytes: size, modelType: "gemma4", isInstruct: instruct) + } + + func test_curatedComeFirstThenOtherCachedModelsSorted() { + let options = MLXModelCatalog.options(cached: [ + cached("zeta/z"), cached("mlx-community/gemma-4-e2b-6bit", size: 42), cached("alpha/a", instruct: true) + ]) + XCTAssertEqual(options.map(\.id), [ + "mlx-community/gemma-4-e4b-5bit", "mlx-community/gemma-4-e2b-6bit", "alpha/a", "zeta/z" + ]) + XCTAssertFalse(options[0].isDownloaded) + XCTAssertTrue(options[0].isRecommended) + XCTAssertTrue(options[1].isDownloaded) + XCTAssertEqual(options[1].sizeBytes, 42, "a downloaded model shows its real size") + XCTAssertEqual(options[2].note, MLXModelCatalog.instructNote) + } + + func test_defaultSelectionPrefersADownloadedCuratedModel() { + XCTAssertNil(MLXModelCatalog.defaultSelection(cached: [])) + XCTAssertEqual(MLXModelCatalog.defaultSelection(cached: [cached("alpha/a")])?.value, "alpha/a") + XCTAssertEqual( + MLXModelCatalog.defaultSelection(cached: [cached("alpha/a"), cached("mlx-community/gemma-4-e2b-6bit")])?.value, + "mlx-community/gemma-4-e2b-6bit" + ) + } + + /// A custom model downloads before it is cached, so no catalog row shows it: the custom field + /// must. A catalog model's download stays on its own row. + func test_downloadWithoutRowIsOnlyACustomModelsDownload() { + let options = MLXModelCatalog.options(cached: []) + let custom = MLXDownloadState(repoID: MLXRepoID("org/custom")!, fraction: 0.4, failure: nil, isActive: true) + XCTAssertEqual(MLXModelCatalog.downloadWithoutRow(custom, options: options), custom) + let failed = MLXDownloadState(repoID: MLXRepoID("org/custom")!, fraction: nil, failure: "No space", isActive: false) + XCTAssertEqual(MLXModelCatalog.downloadWithoutRow(failed, options: options), failed) + let curated = MLXDownloadState( + repoID: MLXModelCatalog.curated[0].repoID, fraction: 0.1, failure: nil, isActive: true) + XCTAssertNil(MLXModelCatalog.downloadWithoutRow(curated, options: options)) + XCTAssertNil(MLXModelCatalog.downloadWithoutRow(nil, options: options)) + } +} + +final class MLXStateTests: XCTestCase { + func test_serverProblemOnlyWhenTheUserMustAct() { + XCTAssertNil(MLXServerState.idle.problem) + XCTAssertNil(MLXServerState.starting(modelID: "a/b").problem) + XCTAssertNil(MLXServerState.ready(port: 1, modelID: "a/b").problem) + XCTAssertNotNil(MLXServerState.unavailable(.notInstalled).problem) + XCTAssertNotNil(MLXServerState.failed(reason: "boom", logTail: []).problem) + } + + func test_notReadyMessageNamesTheReason() { + XCTAssertEqual(MLXServerState.starting(modelID: "a/b").notReadyMessage, "MLX is loading a/b…") + XCTAssertEqual(MLXServerState.unavailable(.noModel).notReadyMessage, "Choose an MLX model in Settings → Engine & Model.") + } + + func test_pathsLiveUnderTheAppsSupportFolder() { + let paths = MLXPaths(root: URL(fileURLWithPath: "/S/Cotabby Dev/MLX")) + XCTAssertEqual(paths.ownServerExecutable.path, "/S/Cotabby Dev/MLX/bin/mlx_lm.server") + XCTAssertEqual(paths.uvExecutable.path, "/S/Cotabby Dev/MLX/uv/uv-aarch64-apple-darwin/uv") + XCTAssertEqual(paths.serverPIDFile.path, "/S/Cotabby Dev/MLX/server.pid") + } +} diff --git a/CotabbyTests/Models/Runtime/Metrics/MenuResourceMonitorTests.swift b/CotabbyTests/Models/Runtime/Metrics/MenuResourceMonitorTests.swift new file mode 100644 index 000000000..d30d9418d --- /dev/null +++ b/CotabbyTests/Models/Runtime/Metrics/MenuResourceMonitorTests.swift @@ -0,0 +1,69 @@ +import XCTest +@testable import Cotabby + +/// Start/stop behavior of the menu bar monitor, driven through `tick()` so no timer has to fire. +/// Instances are kept alive for the process (same isolated-deinit quarantine as the other +/// MainActor stores' tests). +final class MenuResourceMonitorTests: XCTestCase { + @MainActor private static var retained: [MenuResourceMonitor] = [] + + @MainActor + private func makeMonitor(mlx: Bool = false, stops: StopCounter = StopCounter()) -> (MenuResourceMonitor, StopCounter) { + var calls = 0 + let monitor = MenuResourceMonitor( + interval: 3600, + sample: { + calls += 1 + return ResourceUsageSummary( + cpuPercent: Double(calls), memoryBytes: 1, gpuPercent: nil, includesMLXServer: mlx) + }, + primingDelay: 3600, + onStop: { stops.count += 1 } + ) + Self.retained.append(monitor) + return (monitor, stops) + } + + final class StopCounter { var count = 0 } + + @MainActor + func testStartPublishesImmediatelyAndTickUpdates() { + let (monitor, _) = makeMonitor() + XCTAssertNil(monitor.summary) + monitor.start() + XCTAssertNil(monitor.summary, "the priming reading seeds baselines and is not published") + monitor.tick() + XCTAssertEqual(monitor.summary?.cpuPercent, 2, "first published value is the second reading") + monitor.stop() + } + + @MainActor + func testStopEndsTimerAndNotifies() { + let (monitor, stops) = makeMonitor() + monitor.start() + XCTAssertTrue(monitor.isRunning) + monitor.stop() + XCTAssertFalse(monitor.isRunning) + XCTAssertEqual(stops.count, 1) + } + + @MainActor + func testStartTwiceDoesNotResample() { + let (monitor, stops) = makeMonitor() + monitor.start() + monitor.start() + monitor.tick() + XCTAssertEqual(monitor.summary?.cpuPercent, 2, "second start() must not take another priming reading") + monitor.stop() + monitor.stop() + XCTAssertEqual(stops.count, 1, "stop is idempotent") + XCTAssertNil(monitor.summary) + } + + @MainActor + func testMLXFlagPassesThrough() { + let (monitor, _) = makeMonitor(mlx: true) + monitor.tick() + XCTAssertEqual(monitor.summary?.includesMLXServer, true) + } +} diff --git a/CotabbyTests/Models/Settings/SuggestionSettingsMLXTests.swift b/CotabbyTests/Models/Settings/SuggestionSettingsMLXTests.swift new file mode 100644 index 000000000..50f93ffd9 --- /dev/null +++ b/CotabbyTests/Models/Settings/SuggestionSettingsMLXTests.swift @@ -0,0 +1,80 @@ +import XCTest +@testable import Cotabby + +/// MLX settings persistence: the engine raw value, the MLX model ids per power source, and the +/// prompt-cache cap all survive a reload, and a value written by a newer build still loads. +@MainActor +final class SuggestionSettingsMLXTests: XCTestCase { + private static var retained: [AnyObject] = [] + private var suiteNames: [String] = [] + + override func tearDown() { + suiteNames.forEach { UserDefaults.standard.removePersistentDomain(forName: $0) } + suiteNames.removeAll() + super.tearDown() + } + + private func makeDefaults() -> UserDefaults { + let name = "cotabby.test.mlx-settings.\(UUID().uuidString)" + suiteNames.append(name) + return UserDefaults(suiteName: name)! + } + + private func makeModel(_ defaults: UserDefaults) -> SuggestionSettingsModel { + let model = SuggestionSettingsModel(configuration: .standard, userDefaults: defaults) + Self.retained.append(model) + return model + } + + func test_mlxSelectionAndModelSurviveReload() { + let defaults = makeDefaults() + let model = makeModel(defaults) + XCTAssertEqual(model.mlxModelID, "") + XCTAssertEqual(model.mlxPromptCacheBytes, 1 << 30) + + model.selectEngine(.mlx) + model.setMLXModelID("mlx-community/gemma-4-e4b-5bit") + model.setMLXPromptCacheBytes(2 << 30) + + let reloaded = makeModel(defaults) + XCTAssertEqual(reloaded.selectedEngine, .mlx) + XCTAssertEqual(reloaded.mlxModelID, "mlx-community/gemma-4-e4b-5bit") + XCTAssertEqual(reloaded.mlxPromptCacheBytes, 2 << 30) + } + + func test_promptCacheCapIsClamped() { + let model = makeModel(makeDefaults()) + model.setMLXPromptCacheBytes(1) + XCTAssertEqual(model.mlxPromptCacheBytes, 256 << 20) + model.setMLXPromptCacheBytes(64 << 30) + XCTAssertEqual(model.mlxPromptCacheBytes, 8 << 30) + } + + func test_unknownEngineRawValueFallsBackToOpenSource() { + let defaults = makeDefaults() + defaults.set("someFutureEngine", forKey: "cotabbySelectedEngine") + XCTAssertEqual(makeModel(defaults).selectedEngine, .llamaOpenSource) + } + + func test_mlxPowerProfilesRoundTrip() { + let defaults = makeDefaults() + let model = makeModel(defaults) + model.setBatteryProfile(.mlx(modelID: "mlx-community/gemma-4-e2b-6bit")) + model.setPluggedInProfile(.llama(filename: "a.gguf")) + + let reloaded = makeModel(defaults) + XCTAssertEqual(reloaded.batteryProfile, .mlx(modelID: "mlx-community/gemma-4-e2b-6bit")) + XCTAssertEqual(reloaded.pluggedInProfile, .llama(filename: "a.gguf")) + } + + func test_initializePowerProfilesSeedsFromTheActiveMLXModel() { + let model = makeModel(makeDefaults()) + model.initializePowerProfiles( + currentEngine: .mlx, + currentModelFilename: nil, + currentMLXModelID: "mlx-community/gemma-4-e4b-5bit" + ) + XCTAssertEqual(model.batteryProfile, .mlx(modelID: "mlx-community/gemma-4-e4b-5bit")) + XCTAssertEqual(model.pluggedInProfile, .mlx(modelID: "mlx-community/gemma-4-e4b-5bit")) + } +} diff --git a/CotabbyTests/Models/Suggestion/SuggestionEngineKindTests.swift b/CotabbyTests/Models/Suggestion/SuggestionEngineKindTests.swift new file mode 100644 index 000000000..5c4d98a97 --- /dev/null +++ b/CotabbyTests/Models/Suggestion/SuggestionEngineKindTests.swift @@ -0,0 +1,31 @@ +import XCTest +@testable import Cotabby + +/// Pins the two per-engine questions every privacy and speculation gate asks, so adding an engine +/// forces an explicit answer instead of inheriting one from a `!= .openAICompatible` comparison. +final class SuggestionEngineKindTests: XCTestCase { + func test_mlxIsLabelledAndIconned() { + XCTAssertEqual(SuggestionEngineKind.mlx.displayLabel, "MLX") + XCTAssertEqual(SuggestionEngineKind.mlx.systemImageName, "bolt.fill") + XCTAssertFalse(SuggestionEngineKind.mlx.supportsLocalModelManagement) + XCTAssertEqual(SuggestionEngineKind(rawValue: "mlx"), .mlx) + } + + func test_onlyTheEndpointLeavesTheMac() { + XCTAssertTrue(SuggestionEngineKind.appleIntelligence.isOnDevice) + XCTAssertTrue(SuggestionEngineKind.llamaOpenSource.isOnDevice) + XCTAssertTrue(SuggestionEngineKind.mlx.isOnDevice) + XCTAssertFalse(SuggestionEngineKind.openAICompatible.isOnDevice) + } + + func test_onlyInProcessEnginesRunSpeculativeWork() { + XCTAssertTrue(SuggestionEngineKind.appleIntelligence.allowsSpeculativeGeneration) + XCTAssertTrue(SuggestionEngineKind.llamaOpenSource.allowsSpeculativeGeneration) + XCTAssertFalse(SuggestionEngineKind.mlx.allowsSpeculativeGeneration) + XCTAssertFalse(SuggestionEngineKind.openAICompatible.allowsSpeculativeGeneration) + } + + func test_mlxPowerProfileSelectsTheMLXEngine() { + XCTAssertEqual(PowerProfile.mlx(modelID: "org/m").engine, .mlx) + } +} diff --git a/CotabbyTests/Models/Suggestion/SuggestionEngineModelsTests.swift b/CotabbyTests/Models/Suggestion/SuggestionEngineModelsTests.swift index 6e2cd30be..f276da001 100644 --- a/CotabbyTests/Models/Suggestion/SuggestionEngineModelsTests.swift +++ b/CotabbyTests/Models/Suggestion/SuggestionEngineModelsTests.swift @@ -25,7 +25,7 @@ final class SuggestionEngineModelsTests: XCTestCase { // would silently reset every saved engine choice. XCTAssertEqual( SuggestionEngineKind.allCases.map(\.rawValue), - ["appleIntelligence", "llamaOpenSource", "openAICompatible"] + ["appleIntelligence", "llamaOpenSource", "openAICompatible", "mlx"] ) } diff --git a/CotabbyTests/Services/History/TypingHistoryStoreTests.swift b/CotabbyTests/Services/History/TypingHistoryStoreTests.swift index b85c2f2c2..56cb2eeed 100644 --- a/CotabbyTests/Services/History/TypingHistoryStoreTests.swift +++ b/CotabbyTests/Services/History/TypingHistoryStoreTests.swift @@ -188,6 +188,8 @@ final class TypingHistoryStoreTests: XCTestCase { XCTAssertNil(store.phraseContinuation(for: request, engine: .openAICompatible)) XCTAssertEqual(store.historyExamples(for: context, engine: .openAICompatible), []) + // MLX runs on this Mac (127.0.0.1 server Cotabby owns), so history stays available to it. + XCTAssertNotNil(store.phraseContinuation(for: request, engine: .mlx)) store.setUsingHistory(false) XCTAssertNil(store.phraseContinuation(for: request, engine: .appleIntelligence)) diff --git a/CotabbyTests/Services/Runtime/MLX/MLXEngineWiringTests.swift b/CotabbyTests/Services/Runtime/MLX/MLXEngineWiringTests.swift new file mode 100644 index 000000000..a5b9d62fc --- /dev/null +++ b/CotabbyTests/Services/Runtime/MLX/MLXEngineWiringTests.swift @@ -0,0 +1,83 @@ +import XCTest +@testable import Cotabby + +/// File overview: +/// Pins the composition seam between the MLX server manager and the OpenAI-compatible engine: the +/// engine must wait for the server, then send requests to the server's exact port and model. +/// It uses the fake launcher and probe from `MLXServerManagerTests`, so no Python process runs. + +/// The MLX engine is the endpoint client pointed at Cotabby's own server: nothing reaches the +/// network until the server is ready, and then requests carry its exact model id and port. +@MainActor +final class MLXEngineWiringTests: XCTestCase { + private static var retained: [AnyObject] = [] + + override func tearDown() { + MLXWiringStubURLProtocol.handler = nil + MLXWiringStubURLProtocol.requestedURLs = [] + super.tearDown() + } + + func test_engineWaitsForTheServerThenUsesItsPortAndModel() async throws { + let launcher = MLXFakeLauncher() + let probe = MLXFakeProbe() + let manager = MLXServerManager( + paths: MLXPaths(root: FileManager.default.temporaryDirectory.appendingPathComponent("mlx-wire-\(UUID().uuidString)")), + launcher: launcher, probe: probe, pickPort: { 50123 }, sleep: { _ in await Task.yield() }, + verifyListener: { _, _ in true }) + let sessionConfiguration = URLSessionConfiguration.ephemeral + sessionConfiguration.protocolClasses = [MLXWiringStubURLProtocol.self] + let client = OpenAICompatibleAPIClient(session: URLSession(configuration: sessionConfiguration)) + let engine = OpenAICompatibleSuggestionEngine.mlx(client: client, serverManager: manager) + Self.retained.append(contentsOf: [launcher, probe, manager, client, engine] as [AnyObject]) + + await manager.apply(.running(MLXServerLaunch( + executable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), + modelID: "mlx-community/gemma-4-e4b-5bit", promptCacheBytes: 1 << 30))) + do { + _ = try await engine.generateSuggestion(for: CotabbyTestFixtures.suggestionRequest()) + XCTFail("expected unavailable while loading") + } catch SuggestionClientError.unavailable(let message) { + XCTAssertEqual(message, "MLX is loading mlx-community/gemma-4-e4b-5bit…") + } + XCTAssertTrue(MLXWiringStubURLProtocol.requestedURLs.isEmpty) + + await waitUntil { probe.waiting == 1 } + probe.succeedNext() + await waitUntil { if case .ready = manager.state { return true } else { return false } } + MLXWiringStubURLProtocol.handler = { request in + let response = HTTPURLResponse(url: request.url!, statusCode: 200, httpVersion: nil, + headerFields: ["Content-Type": "text/event-stream"])! + return (response, Data("data: {\"choices\":[{\"text\":\" world\"}]}\n\ndata: [DONE]\n\n".utf8)) + } + let result = try await engine.generateSuggestion(for: CotabbyTestFixtures.suggestionRequest()) + XCTAssertEqual(result.rawText, " world") + XCTAssertEqual(MLXWiringStubURLProtocol.requestedURLs, ["http://127.0.0.1:50123/v1/completions"]) + } +} + +private final class MLXWiringStubURLProtocol: URLProtocol { + nonisolated(unsafe) static var handler: ((URLRequest) throws -> (HTTPURLResponse, Data))? + nonisolated(unsafe) static var requestedURLs: [String] = [] + + override class func canInit(with request: URLRequest) -> Bool { true } + override class func canonicalRequest(for request: URLRequest) -> URLRequest { request } + + override func startLoading() { + Self.requestedURLs.append(request.url?.absoluteString ?? "") + guard let handler = Self.handler else { + client?.urlProtocol(self, didFailWithError: URLError(.cannotConnectToHost)) + return + } + do { + let (response, data) = try handler(request) + client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed) + client?.urlProtocol(self, didLoad: data) + client?.urlProtocolDidFinishLoading(self) + } catch { + client?.urlProtocol(self, didFailWithError: error) + } + } + + override func stopLoading() {} +} diff --git a/CotabbyTests/Services/Runtime/MLX/MLXLiveEvalTests.swift b/CotabbyTests/Services/Runtime/MLX/MLXLiveEvalTests.swift new file mode 100644 index 000000000..81d176856 --- /dev/null +++ b/CotabbyTests/Services/Runtime/MLX/MLXLiveEvalTests.swift @@ -0,0 +1,67 @@ +#if RUN_MLX_EVAL +import XCTest +@testable import Cotabby + +/// File overview: +/// The only test that starts a real `mlx_lm.server`. It is compiled and run only with the +/// `RUN_MLX_EVAL` flag, in Release, on a Mac that has mlx-lm and the model in its Hugging Face +/// cache. It walks the same path the app walks: find the toolchain, check the cache, apply the +/// desire, wait for readiness, stream a completion through the engine, then stop and confirm no +/// process is left. Fakes are deliberately absent here, because the point is to prove the real +/// socket check and the real process lifecycle. +@MainActor +final class MLXLiveEvalTests: XCTestCase { + /// Keeps the collaborators alive for the whole run, the same pattern the other MLX test classes + /// use, so no object is released while the server is still starting or generating. + private static var retained: [AnyObject] = [] + + func test_liveServerGeneratesAndStopsCleanly() async throws { + let modelID = ProcessInfo.processInfo.environment["MLX_EVAL_MODEL"] ?? "mlx-community/gemma-4-e4b-5bit" + let paths = MLXPaths(root: FileManager.default.temporaryDirectory.appendingPathComponent("mlx-live-\(UUID().uuidString)")) + let provisioner = MLXToolchainProvisioner.live(paths: MLXPaths.standard()) + let manager = MLXServerManager.live(paths: paths) + let downloader = MLXModelDownloader.live() + let client = OpenAICompatibleAPIClient() + let engine = OpenAICompatibleSuggestionEngine.mlx(client: client, serverManager: manager) + Self.retained.append(contentsOf: [provisioner, manager, downloader, client, engine] as [AnyObject]) + + await provisioner.refresh() + let toolchain = try XCTUnwrap(provisioner.state.toolchain, "mlx-lm is not installed: \(provisioner.state)") + await downloader.refreshCache(toolchain: toolchain) + let desire = MLXServerDesireResolver.desire(for: .init( + engine: .mlx, isAppleSilicon: true, toolchain: provisioner.state, modelID: modelID, + cachedModels: downloader.cachedModels, promptCacheBytes: 1 << 30)) + guard case .running = desire else { return XCTFail("cannot run: \(desire)") } + + let started = Date() + await manager.apply(desire) + await waitUntil(timeout: 150) { + if case .ready = manager.state { return true } + if case .failed = manager.state { return true } + return false + } + guard case .ready = manager.state else { return XCTFail("server: \(manager.state)") } + print("MLX ready after \(Int(Date().timeIntervalSince(started) * 1000)) ms") + let pid = try XCTUnwrap(MLXServerPIDFile(url: paths.serverPIDFile).read()) + + var partials = 0 + var latencies: [Int] = [] + for prefix in ["Thanks for the update. I will send the", "The meeting is moved to Thursday because"] { + let request = CotabbyTestFixtures.suggestionRequest(prefixText: prefix, prompt: prefix) + let begin = Date() + let result = try await engine.generateSuggestion(for: request, onPartial: { _ in partials += 1 }) + latencies.append(Int(Date().timeIntervalSince(begin) * 1000)) + XCTAssertFalse(result.rawText.isEmpty, "empty completion for \(prefix)") + print("MLX: \(prefix)|\(result.rawText)|") + } + print("MLX latencies ms: \(latencies)") + XCTAssertLessThan(latencies[0], 3000, "first suggestion after readiness should not pay the cold-start cost") + XCTAssertGreaterThan(partials, 0, "the stream reported no partials") + + await manager.apply(.stopped) + XCTAssertEqual(manager.state, .idle) + XCTAssertFalse(MLXProcessInspector.isAlive(pid), "mlx_lm.server \(pid) is still running") + try? FileManager.default.removeItem(at: paths.root) + } +} +#endif diff --git a/CotabbyTests/Services/Runtime/MLX/MLXModelDownloaderTests.swift b/CotabbyTests/Services/Runtime/MLX/MLXModelDownloaderTests.swift new file mode 100644 index 000000000..83cc6e25e --- /dev/null +++ b/CotabbyTests/Services/Runtime/MLX/MLXModelDownloaderTests.swift @@ -0,0 +1,272 @@ +import XCTest +@testable import Cotabby + +@MainActor +final class MLXModelDownloaderTests: XCTestCase { + private static var retained: [AnyObject] = [] + private var cacheRoot: URL! + private let toolchain = MLXToolchain( + serverExecutable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), + pythonExecutable: URL(fileURLWithPath: "/mlx/python"), + version: "0.32.0", isCotabbyInstall: true) + + override func setUpWithError() throws { + cacheRoot = FileManager.default.temporaryDirectory.appendingPathComponent("mlx-dl-\(UUID().uuidString)") + try FileManager.default.createDirectory(at: cacheRoot, withIntermediateDirectories: true) + } + + override func tearDownWithError() throws { + try? FileManager.default.removeItem(at: cacheRoot) + } + + /// Writes a complete cached model the way huggingface_hub would leave it. + private func writeCachedModel(_ id: String, type: String = "gemma4") throws { + let repo = cacheRoot.appendingPathComponent(MLXRepoID(id)!.cacheFolderName) + let snapshot = repo.appendingPathComponent("snapshots/rev") + try FileManager.default.createDirectory(at: snapshot, withIntermediateDirectories: true) + try FileManager.default.createDirectory(at: repo.appendingPathComponent("blobs"), withIntermediateDirectories: true) + try FileManager.default.createDirectory(at: repo.appendingPathComponent("refs"), withIntermediateDirectories: true) + try "rev".write(to: repo.appendingPathComponent("refs/main"), atomically: true, encoding: .utf8) + try "{\"model_type\":\"\(type)\"}".write(to: snapshot.appendingPathComponent("config.json"), atomically: true, encoding: .utf8) + try "w".write(to: snapshot.appendingPathComponent("model.safetensors"), atomically: true, encoding: .utf8) + try "{}".write(to: snapshot.appendingPathComponent("tokenizer.json"), atomically: true, encoding: .utf8) + } + + private func makeDownloader( + files: @escaping (String) async throws -> [HFRepoFile] = { _ in [] }, + gated: @escaping (String) async throws -> Bool = { _ in false }, + config: @escaping MLXModelDownloader.ConfigFetch = { _ in Data("{\"model_type\":\"gemma4\"}".utf8) }, + run: @escaping MLXModelDownloader.RunProcess = { _, _, _, _ in MLXProcessResult(status: 0, stdout: "[\"gemma4\"]", stderrTail: []) } + ) -> MLXModelDownloader { + let downloader = MLXModelDownloader( + cacheRoot: cacheRoot, fileManager: .default, + fetchRepoFiles: files, fetchIsGated: gated, fetchConfig: config, runProcess: run, pollInterval: .milliseconds(20)) + Self.retained.append(downloader) + return downloader + } + + func test_refreshFiltersByTheInstalledArchitectures() async throws { + try writeCachedModel("a/gemma", type: "gemma4") + try writeCachedModel("a/other", type: "mamba_x") + let downloader = makeDownloader() + XCTAssertNil(downloader.cachedModels, "nil until the first scan") + await downloader.refreshCache(toolchain: toolchain) + XCTAssertEqual(downloader.cachedModels?.map(\.repoID.value), ["a/gemma"]) + await downloader.refreshCache(toolchain: nil) + XCTAssertEqual(downloader.cachedModels?.count, 2, "without mlx-lm the type filter is skipped") + } + + func test_validationRejectsBadIDsWithoutNetwork() async { + let downloader = makeDownloader(files: { _ in XCTFail("no network for a bad id"); return [] }) + let result = await downloader.validateCustomRepo("../../etc", toolchain: nil) + XCTAssertEqual(result, .invalidID) + } + + func test_validationClassifiesRepos() async { + let missing = makeDownloader(files: { _ in throw HuggingFaceAPIClient.APIError.httpError(statusCode: 404) }) + let missingResult = await missing.validateCustomRepo("a/b", toolchain: nil) + XCTAssertEqual(missingResult, .notFoundOrPrivate(MLXRepoID("a/b")!)) + + let gated = makeDownloader(files: { _ in [HFRepoFile(path: "config.json", size: 1)] }, gated: { _ in true }) + let gatedResult = await gated.validateCustomRepo("a/b", toolchain: nil) + XCTAssertEqual(gatedResult, .gated(MLXRepoID("a/b")!)) + + let gguf = makeDownloader(files: { _ in [HFRepoFile(path: "config.json", size: 1), HFRepoFile(path: "m.gguf", size: 9)] }) + let ggufResult = await gguf.validateCustomRepo("a/b", toolchain: nil) + XCTAssertEqual(ggufResult, .incompatible(MLXRepoID("a/b")!, .missingWeights)) + + let good = makeDownloader(files: { _ in + [HFRepoFile(path: "config.json", size: 1), HFRepoFile(path: "model.safetensors", size: 99), + HFRepoFile(path: "tokenizer.json", size: 1)] + }) + let goodResult = await good.validateCustomRepo("https://huggingface.co/a/b", toolchain: toolchain) + XCTAssertEqual(goodResult, .valid(MLXRepoID("a/b")!, totalBytes: 101, isInstruct: false)) + } + + private let fullTree = [ + HFRepoFile(path: "config.json", size: 1), HFRepoFile(path: "model.safetensors", size: 9), + HFRepoFile(path: "tokenizer.json", size: 1) + ] + + func test_validationRejectsEmbeddingRepo() async { + let tree = fullTree + let d = makeDownloader(files: { _ in tree }) + let result = await d.validateCustomRepo("mlx-community/Qwen3-Embedding-0.6B", toolchain: nil) + XCTAssertEqual(result, .incompatible(MLXRepoID("mlx-community/Qwen3-Embedding-0.6B")!, .embeddingModel)) + } + + func test_validationRejectsUnsupportedModelTypeWhenTheSetIsKnown() async { + let tree = fullTree + let d = makeDownloader( + files: { _ in tree }, + config: { _ in Data("{\"model_type\":\"mamba_x\"}".utf8) }) + let result = await d.validateCustomRepo("a/b", toolchain: toolchain) + XCTAssertEqual(result, .incompatible(MLXRepoID("a/b")!, .unsupportedArchitecture("mamba_x"))) + } + + func test_validationMarksInstructReposAndMapsConfig404() async { + let tree = fullTree + let instruct = makeDownloader(files: { _ in tree }) + let result = await instruct.validateCustomRepo("a/gemma-it", toolchain: toolchain) + XCTAssertEqual(result, .valid(MLXRepoID("a/gemma-it")!, totalBytes: 11, isInstruct: true)) + + let gone = makeDownloader( + files: { _ in tree }, + config: { _ in throw HuggingFaceAPIClient.APIError.httpError(statusCode: 404) }) + let goneResult = await gone.validateCustomRepo("a/b", toolchain: nil) + XCTAssertEqual(goneResult, .incompatible(MLXRepoID("a/b")!, .missingConfig)) + } + + func test_refreshPublishesRejectedModels() async throws { + try writeCachedModel("a/gemma", type: "gemma4") + try writeCachedModel("a/gemma-embed", type: "gemma4") + let downloader = makeDownloader() + await downloader.refreshCache(toolchain: toolchain) + XCTAssertEqual(downloader.rejectedModels, [MLXRepoID("a/gemma-embed")!: .embeddingModel]) + } + + func test_downloadRunsTheToolchainPythonAndRefreshes() async throws { + var calls: [(URL, [String], [String: String])] = [] + let downloader = makeDownloader( + files: { _ in [HFRepoFile(path: "model.safetensors", size: 100)] }, + run: { [unowned self] executable, arguments, environment, _ in + calls.append((executable, arguments, environment)) + if arguments.last == "a/b" { try self.writeCachedModel("a/b") } + return MLXProcessResult(status: 0, stdout: "[\"gemma4\"]", stderrTail: []) + } + ) + downloader.startDownload(MLXRepoID("a/b")!, toolchain: toolchain) + XCTAssertEqual(downloader.download?.isActive, true) + await waitUntil { downloader.download == nil && downloader.cachedModels?.isEmpty == false } + let download = try XCTUnwrap(calls.first { $0.1.last == "a/b" }) + XCTAssertEqual(download.0.path, "/mlx/python") + XCTAssertEqual(download.1, ["-I", "-c", MLXToolchainRules.downloadScript, "a/b"]) + XCTAssertNil(download.2["HF_HUB_OFFLINE"]) + XCTAssertEqual(downloader.cachedModels?.map(\.repoID.value), ["a/b"]) + } + + func test_downloadFailureKeepsTheLastErrorLine() async { + let downloader = makeDownloader(run: { _, arguments, _, _ in + arguments.last == "a/b" + ? MLXProcessResult(status: 1, stdout: "", stderrTail: ["Traceback", "RepositoryNotFoundError: a/b"]) + : MLXProcessResult(status: 0, stdout: "[]", stderrTail: []) + }) + downloader.startDownload(MLXRepoID("a/b")!, toolchain: toolchain) + await waitUntil { downloader.download?.isActive == false } + XCTAssertEqual(downloader.download?.failure, "RepositoryNotFoundError: a/b") + } + + func test_finishedDownloadOfUnsupportedArchitectureReportsIt() async { + let downloader = makeDownloader(run: { [unowned self] _, arguments, _, _ in + if arguments.last == "a/b" { try self.writeCachedModel("a/b", type: "mamba_x") } + return MLXProcessResult(status: 0, stdout: "[\"gemma4\"]", stderrTail: []) + }) + downloader.startDownload(MLXRepoID("a/b")!, toolchain: toolchain) + await waitUntil { downloader.download?.isActive == false } + XCTAssertEqual(downloader.download?.failure, "Downloaded, but it can't be used. Architecture “mamba_x” isn't supported by the installed mlx-lm.") + XCTAssertEqual(downloader.cachedModels?.isEmpty, true) + } + + func test_progressAndCancel() async throws { + let downloader = makeDownloader( + files: { _ in [HFRepoFile(path: "model.safetensors", size: 1000)] }, + run: { [unowned self] _, arguments, _, _ in + guard arguments.last == "a/b" else { return MLXProcessResult(status: 0, stdout: "[]", stderrTail: []) } + let blobs = self.cacheRoot.appendingPathComponent("models--a--b/blobs") + try FileManager.default.createDirectory(at: blobs, withIntermediateDirectories: true) + try Data(count: 500).write(to: blobs.appendingPathComponent("x.incomplete")) + try await Task.sleep(for: .seconds(60)) + return MLXProcessResult(status: 0, stdout: "", stderrTail: []) + } + ) + downloader.startDownload(MLXRepoID("a/b")!, toolchain: toolchain) + await waitUntil { downloader.download?.fraction == 0.5 } + downloader.startDownload(MLXRepoID("c/d")!, toolchain: toolchain) + XCTAssertEqual(downloader.download?.repoID.value, "a/b", "one download at a time") + downloader.cancelDownload() + await waitUntil { downloader.download == nil } + } + + /// Review focus 2. + func test_deletingActiveModelRefreshesCacheWithoutIt() async throws { + try writeCachedModel("a/b") + let downloader = makeDownloader() + await downloader.refreshCache(toolchain: toolchain) + XCTAssertEqual(downloader.cachedModels?.count, 1) + await downloader.delete(MLXRepoID("a/b")!, toolchain: toolchain) + XCTAssertEqual(downloader.cachedModels, []) + XCTAssertFalse(FileManager.default.fileExists(atPath: cacheRoot.appendingPathComponent("models--a--b").path)) + } + + /// Review focus 2 with a race: a scan that read the disk before the delete finishes after it. + /// The first scan reads the model, then holds; the delete's own scan runs and publishes the + /// list without it; releasing the stale scan must not bring the model back. + func test_staleRefreshCannotResurrectADeletedModel() async throws { + try writeCachedModel("a/b") + let calls = CallCounter() + let (entered, enteredSignal) = AsyncStream.makeStream() + let (release, releaseSignal) = AsyncStream.makeStream() + let downloader = MLXModelDownloader( + cacheRoot: cacheRoot, fileManager: .default, + fetchRepoFiles: { _ in [] }, fetchIsGated: { _ in false }, + runProcess: { _, _, _, _ in MLXProcessResult(status: 0, stdout: "[\"gemma4\"]", stderrTail: []) }, + pollInterval: .milliseconds(20), + scanCache: { root, types in + let models = MLXCacheScanner.scanDetailed(cacheRoot: root, supportedModelTypes: types) + if calls.next() == 1 { + enteredSignal.yield() + for await _ in release { break } + } + return models + } + ) + let stale = Task { await downloader.refreshCache(toolchain: nil) } + for await _ in entered { break } + + // Nil toolchain skips the type probe, so the delete's scan does not wait on the held scan. + await downloader.delete(MLXRepoID("a/b")!, toolchain: nil) + XCTAssertEqual(downloader.cachedModels, []) + + releaseSignal.yield() + await stale.value + XCTAssertEqual(downloader.cachedModels, [], "the older scan saw the model before the delete and must not publish it") + } + + /// Review focus 2: deleting the repo that is downloading must stop the download first, so the + /// partial files cannot reappear once the folder is gone. + func test_deletingTheActiveDownloadCancelsItFirst() async throws { + let downloader = makeDownloader( + files: { _ in [HFRepoFile(path: "model.safetensors", size: 1000)] }, + run: { [unowned self] _, arguments, _, _ in + guard arguments.last == "a/b" else { return MLXProcessResult(status: 0, stdout: "[]", stderrTail: []) } + let blobs = self.cacheRoot.appendingPathComponent("models--a--b/blobs") + try FileManager.default.createDirectory(at: blobs, withIntermediateDirectories: true) + try Data(count: 500).write(to: blobs.appendingPathComponent("x.incomplete")) + try await Task.sleep(for: .seconds(60)) + return MLXProcessResult(status: 0, stdout: "", stderrTail: []) + } + ) + downloader.startDownload(MLXRepoID("a/b")!, toolchain: toolchain) + await waitUntil { downloader.download?.fraction == 0.5 } + + await downloader.delete(MLXRepoID("a/b")!, toolchain: nil) + + XCTAssertNil(downloader.download, "the delete waited for the download to stop") + XCTAssertFalse(FileManager.default.fileExists(atPath: cacheRoot.appendingPathComponent("models--a--b").path)) + XCTAssertEqual(downloader.cachedModels, []) + } +} + +/// Counts calls from a `@Sendable` closure. The lock makes the count safe to bump from the detached scan. +private final class CallCounter: @unchecked Sendable { + private let lock = NSLock() + private var count = 0 + + /// Returns the 1-based number of this call. + func next() -> Int { + lock.lock() + defer { lock.unlock() } + count += 1 + return count + } +} diff --git a/CotabbyTests/Services/Runtime/MLX/MLXProcessRunnerTests.swift b/CotabbyTests/Services/Runtime/MLX/MLXProcessRunnerTests.swift new file mode 100644 index 000000000..5aafc5c3c --- /dev/null +++ b/CotabbyTests/Services/Runtime/MLX/MLXProcessRunnerTests.swift @@ -0,0 +1,124 @@ +import XCTest +@testable import Cotabby + +/// Real short-lived processes (/bin/sh, /bin/sleep): exit status, output capture, timeouts, +/// cancellation, and a child that writes more than a pipe buffer holds. +final class MLXProcessRunnerTests: XCTestCase { + private let sh = URL(fileURLWithPath: "/bin/sh") + + func test_returnsStatusAndStdout() async throws { + let result = try await MLXProcessRunner.run( + executable: sh, arguments: ["-c", "printf hello; exit 3"], environment: [:], timeout: .seconds(5)) + XCTAssertEqual(result.status, 3) + XCTAssertEqual(result.stdout, "hello") + } + + func test_keepsTheLastStderrLines() async throws { + let result = try await MLXProcessRunner.run( + executable: sh, + arguments: ["-c", "i=1; while [ $i -le 30 ]; do echo line$i >&2; i=$((i+1)); done"], + environment: [:], timeout: .seconds(5)) + XCTAssertEqual(result.stderrTail.count, 20) + XCTAssertEqual(result.stderrTail.first, "line11") + XCTAssertEqual(result.stderrTail.last, "line30") + } + + func test_passesTheEnvironment() async throws { + let result = try await MLXProcessRunner.run( + executable: sh, arguments: ["-c", "printf \"$FOO\""], environment: ["FOO": "bar"], timeout: .seconds(5)) + XCTAssertEqual(result.stdout, "bar") + } + + func test_largeOutputDoesNotDeadlockAndIsCapped() async throws { + let result = try await MLXProcessRunner.run( + executable: sh, arguments: ["-c", "head -c 3000000 /dev/zero | tr '\\0' a"], + environment: [:], timeout: .seconds(20)) + XCTAssertEqual(result.status, 0) + XCTAssertEqual(result.stdout.utf8.count, 1 << 20) + } + + func test_timeoutTerminatesTheChild() async { + let start = Date() + do { + _ = try await MLXProcessRunner.run( + executable: URL(fileURLWithPath: "/bin/sleep"), arguments: ["30"], environment: [:], + timeout: .milliseconds(300)) + XCTFail("expected a timeout") + } catch MLXProcessRunnerError.timedOut { + XCTAssertLessThan(Date().timeIntervalSince(start), 5) + } catch { + XCTFail("unexpected \(error)") + } + } + + func test_cancellationTerminatesTheChild() async { + let task = Task { + try await MLXProcessRunner.run( + executable: URL(fileURLWithPath: "/bin/sleep"), arguments: ["30"], environment: [:], + timeout: .seconds(60)) + } + try? await Task.sleep(for: .milliseconds(200)) + let start = Date() + task.cancel() + do { + _ = try await task.value + XCTFail("expected cancellation") + } catch is CancellationError { + XCTAssertLessThan(Date().timeIntervalSince(start), 5) + } catch { + XCTFail("unexpected \(error)") + } + } + + /// A grandchild that keeps the pipe open must not stretch the run past the timeout grace. + func test_grandchildHoldingThePipeDoesNotBlock() async { + let start = Date() + do { + _ = try await MLXProcessRunner.run( + executable: sh, arguments: ["-c", "sleep 30 & echo started; sleep 30"], environment: [:], + timeout: .milliseconds(500)) + XCTFail("expected a timeout") + } catch MLXProcessRunnerError.timedOut { + XCTAssertLessThan(Date().timeIntervalSince(start), 6) + } catch { + XCTFail("unexpected \(error)") + } + } + + func test_missingExecutableIsALaunchFailure() async { + do { + _ = try await MLXProcessRunner.run( + executable: URL(fileURLWithPath: "/nonexistent/tool"), arguments: [], environment: [:], + timeout: .seconds(1)) + XCTFail("expected a launch failure") + } catch MLXProcessRunnerError.launchFailed { + } catch { + XCTFail("unexpected \(error)") + } + } +} + +/// The collector splits stderr into lines as bytes, so pipe reads that cut a character in half +/// or redraw a progress line with `\r` still give whole, readable lines. +final class MLXOutputCollectorTests: XCTestCase { + func test_characterSplitAcrossReadsIsDecodedWhole() { + let collector = MLXOutputCollector(tailLines: 5) + let bytes = Array("Grüße\n".utf8) + // "ü" is two bytes; the first read ends between them. + collector.appendStderr(Data(bytes[..<3])) + collector.appendStderr(Data(bytes[3...])) + XCTAssertEqual(collector.stderrTail, ["Grüße"]) + } + + func test_carriageReturnsSplitLinesAndTheUnterminatedTailIsKept() { + let collector = MLXOutputCollector(tailLines: 5) + collector.appendStderr(Data("10%\r50%\r\nDone\npartial".utf8)) + XCTAssertEqual(collector.stderrTail, ["10%", "50%", "Done", "partial"]) + } + + func test_invalidUTF8IsReportedBySize() { + let collector = MLXOutputCollector(tailLines: 5) + collector.appendStderr(Data([0xFF, 0xFE, UInt8(ascii: "\n")])) + XCTAssertEqual(collector.stderrTail, ["<2 bytes of non-UTF-8 output>"]) + } +} diff --git a/CotabbyTests/Services/Runtime/MLX/MLXServerManagerTests.swift b/CotabbyTests/Services/Runtime/MLX/MLXServerManagerTests.swift new file mode 100644 index 000000000..3bbfeaec1 --- /dev/null +++ b/CotabbyTests/Services/Runtime/MLX/MLXServerManagerTests.swift @@ -0,0 +1,572 @@ +import Darwin +import XCTest +@testable import Cotabby + +@MainActor +final class MLXServerManagerTests: XCTestCase { + private static var retained: [AnyObject] = [] + private var root: URL! + private var paths: MLXPaths! + private var launcher: MLXFakeLauncher! + private var probe: MLXFakeProbe! + private var sleeps: [Duration] = [] + private var clock = Date(timeIntervalSince1970: 1_000_000) + + private let launchA = MLXServerLaunch( + executable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), modelID: "mlx-community/a", promptCacheBytes: 1 << 30) + private let launchB = MLXServerLaunch( + executable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), modelID: "mlx-community/b", promptCacheBytes: 1 << 30) + private let launchC = MLXServerLaunch( + executable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), modelID: "mlx-community/c", promptCacheBytes: 1 << 30) + + override func setUp() { + super.setUp() + root = FileManager.default.temporaryDirectory.appendingPathComponent("mlx-server-\(UUID().uuidString)") + paths = MLXPaths(root: root) + launcher = MLXFakeLauncher() + probe = MLXFakeProbe() + sleeps = [] + Self.retained.append(contentsOf: [launcher!, probe!] as [AnyObject]) + } + + override func tearDown() { + try? FileManager.default.removeItem(at: root) + super.tearDown() + } + + /// `sleep` overrides the recording, yielding sleep. A sleep that never returns keeps a restart + /// pending, which lets a test freeze the state between a crash and its restart. + private func makeManager( + sleep: MLXServerManager.Sleep? = nil, + verifyListener: @escaping (Int32, Int) -> Bool = { _, _ in true } + ) -> MLXServerManager { + let manager = MLXServerManager( + paths: paths, + launcher: launcher, + probe: probe, + pickPort: { 50123 }, + sleep: sleep ?? { [unowned self] duration in + self.sleeps.append(duration) + await Task.yield() + }, + now: { [unowned self] in self.clock }, + baseEnvironment: ["HOME": "/Users/u", "PYTHONPATH": "/evil"], + verifyListener: verifyListener + ) + Self.retained.append(manager) + return manager + } + + private func isReady(_ manager: MLXServerManager) -> Bool { + if case .ready = manager.state { return true } + return false + } + + private func startReady(_ manager: MLXServerManager, _ launch: MLXServerLaunch) async { + await manager.apply(.running(launch)) + await waitUntil { self.probe.waiting >= 1 } + probe.succeedNext() + await waitUntil { self.isReady(manager) } + } + + func test_startLaunchesTheExactCommandAndBecomesReady() async throws { + let manager = makeManager() + await manager.apply(.running(launchA)) + XCTAssertEqual(manager.state, .starting(modelID: "mlx-community/a")) + let launch = try XCTUnwrap(launcher.launches.first) + XCTAssertEqual(launch.executable.path, "/mlx/bin/mlx_lm.server") + XCTAssertEqual(launch.arguments, MLXToolchainRules.serverArguments( + modelID: "mlx-community/a", port: 50123, promptCacheBytes: 1 << 30)) + XCTAssertEqual(launch.environment["HF_HUB_OFFLINE"], "1") + XCTAssertNil(launch.environment["PYTHONPATH"]) + XCTAssertEqual(MLXServerPIDFile(url: paths.serverPIDFile).read(), 1000) + + XCTAssertThrowsError(try manager.readyConfiguration) { error in + XCTAssertEqual(error.localizedDescription, "MLX is loading mlx-community/a…") + } + await waitUntil { self.probe.waiting == 1 } + probe.succeedNext() + await waitUntil { manager.state == .ready(port: 50123, modelID: "mlx-community/a") } + let configuration = try manager.readyConfiguration + XCTAssertEqual(configuration.baseURL.absoluteString, "http://127.0.0.1:50123/v1") + XCTAssertEqual(configuration.modelName, "mlx-community/a") + XCTAssertEqual(configuration.apiMode, .completions) + } + + func test_sameDesireAgainDoesNotRelaunch() async { + let manager = makeManager() + await startReady(manager, launchA) + await manager.apply(.running(launchA)) + XCTAssertEqual(launcher.launches.count, 1) + } + + func test_modelChangeStopsTheOldServerAndStartsTheNew() async { + let manager = makeManager() + await startReady(manager, launchA) + await manager.apply(.running(launchB)) + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1) + XCTAssertEqual(launcher.launches.count, 2) + XCTAssertEqual(manager.state, .starting(modelID: "mlx-community/b")) + } + + func test_stopSendsTermThenKillAfterTheGracePeriod() async { + launcher.exitsOnTerminate = false + let manager = makeManager() + await startReady(manager, launchA) + await manager.apply(.stopped) + let process = launcher.launches[0].process + XCTAssertEqual(process.terminateCount, 1) + XCTAssertEqual(process.killCount, 1) + XCTAssertEqual(sleeps, Array(repeating: .milliseconds(100), count: 30)) + XCTAssertEqual(manager.state, .idle) + XCTAssertNil(MLXServerPIDFile(url: paths.serverPIDFile).read()) + } + + func test_aCooperativeServerIsNotKilled() async { + let manager = makeManager() + await startReady(manager, launchA) + await manager.apply(.stopped) + XCTAssertEqual(launcher.launches[0].process.killCount, 0) + XCTAssertTrue(sleeps.isEmpty) + } + + func test_crashesRestartWithBackoffAndTheThirdWithinTwoMinutesFails() async { + let manager = makeManager() + await startReady(manager, launchA) + + launcher.crash(0) + await waitUntil { self.launcher.launches.count == 2 } + XCTAssertEqual(sleeps, [.seconds(1)]) + + clock += 10 + launcher.crash(1) + await waitUntil { self.launcher.launches.count == 3 } + XCTAssertEqual(sleeps, [.seconds(1), .seconds(5)]) + + clock += 10 + launcher.launches[2].process.tail = ["Traceback", "boom"] + launcher.crash(2) + await waitUntil { if case .failed = manager.state { return true } else { return false } } + XCTAssertEqual(manager.state, .failed(reason: "MLX stopped unexpectedly (exit status 1).", + logTail: ["Traceback", "boom"])) + for _ in 0..<5 { await Task.yield() } + XCTAssertEqual(launcher.launches.count, 3, "no restarts after failing") + } + + func test_threeConsecutiveSlowFailuresFail() async { + let manager = makeManager() + await startReady(manager, launchA) + for index in 0..<3 { + clock += 121 + launcher.crash(index) + if index < 2 { await waitUntil { self.launcher.launches.count == index + 2 } } + } + await waitUntil { if case .failed = manager.state { return true } else { return false } } + XCTAssertEqual(sleeps, [.seconds(1), .seconds(5)]) + XCTAssertEqual(launcher.launches.count, 3, "no restart after the third consecutive failure") + } + + func test_retryAfterFailureStartsAgain() async { + let manager = makeManager() + await startReady(manager, launchA) + for index in 0..<3 { + launcher.crash(index) + if index < 2 { await waitUntil { self.launcher.launches.count == index + 2 } } + } + await waitUntil { if case .failed = manager.state { return true } else { return false } } + await manager.apply(.running(launchA)) + XCTAssertEqual(launcher.launches.count, 3, "the same desire does not leave the failed state") + await manager.retry() + XCTAssertEqual(launcher.launches.count, 4) + XCTAssertEqual(manager.state, .starting(modelID: "mlx-community/a")) + } + + /// Review focus 1. + func test_stopWhileStartingTerminatesAndIgnoresLateReadiness() async { + let manager = makeManager() + await manager.apply(.running(launchA)) + await waitUntil { self.probe.waiting == 1 } + await manager.apply(.stopped) + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1) + probe.succeedNext() + for _ in 0..<10 { await Task.yield() } + XCTAssertEqual(manager.state, .idle) + } + + func test_readinessFailureTerminatesAndRestarts() async { + let manager = makeManager() + await manager.apply(.running(launchA)) + await waitUntil { self.probe.waiting == 1 } + probe.failNext() + await waitUntil { self.launcher.launches.count == 2 } + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1) + XCTAssertEqual(sleeps, [.seconds(1)]) + } + + func test_unavailableDesirePublishesTheReasonWithoutLaunching() async { + let manager = makeManager() + await manager.apply(.unavailable(.noModel)) + XCTAssertEqual(manager.state, .unavailable(.noModel)) + XCTAssertTrue(launcher.launches.isEmpty) + XCTAssertThrowsError(try manager.readyConfiguration) { error in + XCTAssertEqual(error.localizedDescription, MLXUnavailableReason.noModel.message) + } + } + + func test_shutdownSyncKillsAStubbornServer() async { + launcher.exitsOnTerminate = false + let manager = makeManager() + await startReady(manager, launchA) + manager.shutdownSync(timeoutSeconds: 0.05) + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1) + XCTAssertEqual(launcher.launches[0].process.killCount, 1) + XCTAssertEqual(manager.state, .idle) + } + + func test_orphanCleanupOnlySignalsOurOwnServer() async { + let manager = makeManager() + let pidFile = MLXServerPIDFile(url: paths.serverPIDFile) + var signals: [Int32] = [] + + pidFile.write(pid: 4242) + await manager.cleanUpOrphan(isAlive: { _ in true }, arguments: { _ in ["/usr/bin/vim"] }, + signal: { _, signal in signals.append(signal) }) + XCTAssertEqual(signals, []) + XCTAssertNil(pidFile.read()) + + pidFile.write(pid: 4242) + await manager.cleanUpOrphan( + isAlive: { _ in true }, + arguments: { _ in ["/p/python", "/x/MLX/bin/mlx_lm.server", "--host", "127.0.0.1", "--port", "9"] }, + signal: { _, signal in signals.append(signal) }) + XCTAssertEqual(signals, [SIGTERM, SIGKILL]) + XCTAssertNil(pidFile.read()) + } + + func test_loopbackPortIsUsable() throws { + let port = try MLXLoopbackPort.next() + XCTAssertTrue((1024...65535).contains(port)) + } + + /// Review fix 1: a crash clears `current` while the probe is still suspended. The probe's later + /// success must not publish `.ready` for a process that no longer exists. + func test_lateReadinessAfterACrashIsIgnored() async { + let manager = makeManager(sleep: { _ in try await Task.sleep(for: .seconds(3600)) }) + await manager.apply(.running(launchA)) + await waitUntil { self.probe.waiting == 1 } + launcher.crash(0) + probe.succeedNext() + for _ in 0..<10 { await Task.yield() } + XCTAssertEqual(manager.state, .starting(modelID: "mlx-community/a")) + XCTAssertEqual(launcher.launches.count, 1, "the restart is still pending, so nothing relaunched") + } + + /// Review fix 2: stopping A waits out its grace period. Meanwhile C starts and writes its pid. + /// A's late cleanup must leave C's pid file alone. + func test_stoppingAnOldServerKeepsTheNewServersPidFile() async { + launcher.exitsOnTerminate = false + let gate = SleepGate() + let manager = makeManager(sleep: { _ in await gate.pause() }) + await startReady(manager, launchA) + let stopping = Task { await manager.apply(.running(launchB)) } + await waitUntil { gate.count == 1 } + // A is frozen inside its grace period. C takes over, then A exits and its cleanup runs. + await manager.apply(.running(launchC)) + launcher.launches[0].process.isRunning = false + gate.releaseAll() + await stopping.value + + XCTAssertEqual(launcher.launches.count, 2, "B was superseded before it launched") + XCTAssertEqual(launcher.launches[1].arguments[1], "mlx-community/c") + XCTAssertEqual(MLXServerPIDFile(url: paths.serverPIDFile).read(), launcher.launches[1].process.processIdentifier) + } + + /// Review fix 3: the pid file names the live server, so orphan cleanup must not treat it as one. + func test_orphanCleanupLeavesTheCurrentServersPidFile() async { + let manager = makeManager() + await startReady(manager, launchA) + var signals: [Int32] = [] + await manager.cleanUpOrphan( + isAlive: { _ in true }, + arguments: { _ in ["/p/python", "/x/MLX/bin/mlx_lm.server", "--host", "127.0.0.1"] }, + signal: { _, signal in signals.append(signal) }) + XCTAssertEqual(signals, []) + XCTAssertEqual(MLXServerPIDFile(url: paths.serverPIDFile).read(), 1000) + } + + /// Review fix 4: app quit during a stop must still reach the process being stopped. + func test_shutdownSyncKillsAServerThatIsStillStopping() async { + launcher.exitsOnTerminate = false + let gate = SleepGate() + let manager = makeManager(sleep: { _ in await gate.pause() }) + await startReady(manager, launchA) + let stopping = Task { await manager.apply(.stopped) } + await waitUntil { gate.count == 1 } + manager.shutdownSync(timeoutSeconds: 0.05) + XCTAssertEqual(launcher.launches[0].process.killCount, 1, "only shutdown can kill it: the stop is frozen") + gate.releaseAll() + await stopping.value + XCTAssertEqual(launcher.launches[0].process.killCount, 1, "the resumed stop does not kill it twice") + XCTAssertEqual(manager.state, .idle) + } + + /// Review fix 6: a readiness answer from a process that does not own the port is refused. + func test_readinessFromAForeignListenerIsRejected() async { + let manager = makeManager(verifyListener: { _, _ in false }) + await manager.apply(.running(launchA)) + await waitUntil { self.probe.waiting == 1 } + probe.succeedNext() + await waitUntil { self.launcher.launches[0].process.terminateCount == 1 } + XCTAssertFalse(isReady(manager)) + XCTAssertEqual(manager.state, .starting(modelID: "mlx-community/a"), "a restart follows the failure") + } + + /// Proves the libproc port check against a real socket this test process owns. + func test_ownsTCPSocketFindsOurOwnListener() throws { + let listener = socket(AF_INET, SOCK_STREAM, 0) + XCTAssertGreaterThanOrEqual(listener, 0) + defer { close(listener) } + var address = sockaddr_in() + address.sin_len = UInt8(MemoryLayout.size) + address.sin_family = sa_family_t(AF_INET) + address.sin_port = 0 + address.sin_addr.s_addr = inet_addr("127.0.0.1") + let bound = withUnsafeMutablePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { + Darwin.bind(listener, $0, socklen_t(MemoryLayout.size)) + } + } + XCTAssertEqual(bound, 0) + XCTAssertEqual(listen(listener, 1), 0) + var length = socklen_t(MemoryLayout.size) + _ = withUnsafeMutablePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(listener, $0, &length) } + } + let port = Int(UInt16(bigEndian: address.sin_port)) + + XCTAssertTrue(MLXProcessInspector.ownsTCPSocket(pid: getpid(), port: port)) + XCTAssertFalse(MLXProcessInspector.ownsTCPSocket(pid: getpid(), port: port == 1 ? 2 : 1)) + } +} + +/// Holds each stop-grace sleep until the test releases it, so a stop can be frozen mid-grace. +@MainActor +final class SleepGate { + private var waiting: [CheckedContinuation] = [] + var count: Int { waiting.count } + + func pause() async { + await withCheckedContinuation { waiting.append($0) } + } + + func releaseAll() { + let released = waiting + waiting = [] + released.forEach { $0.resume() } + } +} + +// The three doubles below are internal (not private) so `MLXEngineWiringTests` can reuse them. + +@MainActor +final class MLXFakeProcess: MLXRunningProcess { + let processIdentifier: Int32 + var isRunning = true + var exitsOnTerminate = true + var tail: [String] = [] + private(set) var terminateCount = 0 + private(set) var killCount = 0 + + init(pid: Int32) { processIdentifier = pid } + + func terminate() { + terminateCount += 1 + if exitsOnTerminate { isRunning = false } + } + + func forceKill() { + killCount += 1 + isRunning = false + } + + func stderrTail() -> [String] { tail } +} + +@MainActor +final class MLXFakeLauncher: MLXProcessLaunching { + struct Launch { + let executable: URL + let arguments: [String] + let environment: [String: String] + let process: MLXFakeProcess + let onExit: @MainActor (Int32) -> Void + } + + private(set) var launches: [Launch] = [] + var exitsOnTerminate = true + + func launch( + executable: URL, + arguments: [String], + environment: [String: String], + onExit: @escaping @MainActor (Int32) -> Void + ) throws -> any MLXRunningProcess { + let process = MLXFakeProcess(pid: Int32(1000 + launches.count)) + process.exitsOnTerminate = exitsOnTerminate + launches.append(Launch(executable: executable, arguments: arguments, environment: environment, + process: process, onExit: onExit)) + return process + } + + /// Simulates launch `index` dying on its own. + func crash(_ index: Int, status: Int32 = 1) { + launches[index].process.isRunning = false + launches[index].onExit(status) + } +} + +/// Each readiness wait suspends until the test resolves it, so tests control exactly when (and +/// whether) a server becomes ready, including after it was stopped. +@MainActor +final class MLXFakeProbe: MLXReadinessProbing { + private var pending: [CheckedContinuation] = [] + var waiting: Int { pending.count } + + func waitUntilReady(port: Int, modelID: String, timeout: Duration) async throws { + try await withCheckedThrowingContinuation { pending.append($0) } + } + + func succeedNext() { pending.removeFirst().resume() } + func failNext() { pending.removeFirst().resume(throwing: MLXServerError.readinessTimedOut) } +} + +// MARK: - reportBroken (generator died while the process stayed up) + +@MainActor +final class MLXServerManagerBrokenGeneratorTests: XCTestCase { + private static var retained: [AnyObject] = [] + private var root: URL! + private var launcher: MLXFakeLauncher! + private var probe: MLXFakeProbe! + private var sleeps: [Duration] = [] + private let launch = MLXServerLaunch( + executable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), modelID: "mlx-community/a", promptCacheBytes: 1 << 30) + + override func setUp() { + super.setUp() + root = FileManager.default.temporaryDirectory.appendingPathComponent("mlx-broken-\(UUID().uuidString)") + launcher = MLXFakeLauncher() + probe = MLXFakeProbe() + sleeps = [] + Self.retained.append(contentsOf: [launcher!, probe!] as [AnyObject]) + } + + override func tearDown() { + try? FileManager.default.removeItem(at: root) + super.tearDown() + } + + private func makeManager() -> MLXServerManager { + let manager = MLXServerManager( + paths: MLXPaths(root: root), launcher: launcher, probe: probe, pickPort: { 50123 }, + sleep: { [unowned self] duration in self.sleeps.append(duration); await Task.yield() }, + now: { Date(timeIntervalSince1970: 1_000_000) }, baseEnvironment: [:], + verifyListener: { _, _ in true }) + Self.retained.append(manager) + return manager + } + + private func isReady(_ manager: MLXServerManager) -> Bool { + if case .ready = manager.state { return true } + return false + } + + private func startReady(_ manager: MLXServerManager) async { + await manager.apply(.running(launch)) + await waitUntil { self.probe.waiting >= 1 } + probe.succeedNext() + await waitUntil { self.isReady(manager) } + } + + private func becomeReadyAgain(_ manager: MLXServerManager, launches: Int) async { + await waitUntil { self.launcher.launches.count == launches } + await waitUntil { self.probe.waiting >= 1 } + probe.succeedNext() + await waitUntil { self.isReady(manager) } + } + + func test_reportBrokenTerminatesAndRelaunchesAfterOneSecond() async { + let manager = makeManager() + await startReady(manager) + manager.reportBroken(port: 50123, reason: "HTTP 404") + await waitUntil { self.launcher.launches.count == 2 } + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1) + XCTAssertEqual(sleeps.first, .seconds(1)) + XCTAssertEqual(MLXServerPIDFile(url: MLXPaths(root: root).serverPIDFile).read(), 1001) + // The old process's exit callback must not count a second failure. + launcher.launches[0].onExit(15) + for _ in 0..<5 { await Task.yield() } + XCTAssertEqual(launcher.launches.count, 2) + } + + func test_thirdReportFailsWithTheReasonAndTail() async { + let manager = makeManager() + await startReady(manager) + manager.reportBroken(port: 50123, reason: "HTTP 404") + await becomeReadyAgain(manager, launches: 2) + manager.reportBroken(port: 50123, reason: "HTTP 404") + await becomeReadyAgain(manager, launches: 3) + launcher.launches[2].process.tail = ["mlx_lm.server generation thread died: boom"] + manager.reportBroken(port: 50123, reason: "HTTP 404") + await waitUntil { if case .failed = manager.state { return true } else { return false } } + XCTAssertEqual(manager.state, .failed( + reason: "MLX's generator stopped working: HTTP 404", + logTail: ["mlx_lm.server generation thread died: boom"])) + XCTAssertEqual(sleeps.filter { $0 == .seconds(1) }.count, 2) + } + + func test_aReportForAnotherPortIsIgnored() async { + let manager = makeManager() + await startReady(manager) + manager.reportBroken(port: 49999, reason: "HTTP 404") + for _ in 0..<5 { await Task.yield() } + XCTAssertTrue(isReady(manager)) + XCTAssertEqual(launcher.launches[0].process.terminateCount, 0) + } + + func test_aReportWhileStartingIsIgnored() async { + let manager = makeManager() + await manager.apply(.running(launch)) + XCTAssertEqual(manager.state, .starting(modelID: "mlx-community/a")) + manager.reportBroken(port: 50123, reason: "HTTP 404") + for _ in 0..<5 { await Task.yield() } + XCTAssertEqual(launcher.launches[0].process.terminateCount, 0) + XCTAssertEqual(manager.state, .starting(modelID: "mlx-community/a")) + } + + /// A quit that lands before the report's task runs must still reach the broken process: the + /// report signals it and records it as stopping without suspending first. + func test_quitRightAfterAReportStillStopsTheBrokenServer() async { + launcher.exitsOnTerminate = false + let manager = makeManager() + await startReady(manager) + manager.reportBroken(port: 50123, reason: "HTTP 404") + manager.shutdownSync(timeoutSeconds: 0.05) + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1) + XCTAssertEqual(launcher.launches[0].process.killCount, 1, "shutdown SIGKILLs the stubborn process") + for _ in 0..<5 { await Task.yield() } + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1, "SIGTERM is sent once") + XCTAssertEqual(launcher.launches.count, 1, "a quit leaves nothing to restart") + XCTAssertEqual(manager.state, .idle) + } + + func test_aSecondReportDuringTheRestartDoesNotCountAgain() async { + let manager = makeManager() + await startReady(manager) + manager.reportBroken(port: 50123, reason: "HTTP 404") + manager.reportBroken(port: 50123, reason: "HTTP 404") + await waitUntil { self.launcher.launches.count == 2 } + XCTAssertEqual(launcher.launches[0].process.terminateCount, 1) + XCTAssertEqual(sleeps.filter { $0 == .seconds(1) }.count, 1) + } +} diff --git a/CotabbyTests/Services/Runtime/MLX/MLXToolchainProvisionerTests.swift b/CotabbyTests/Services/Runtime/MLX/MLXToolchainProvisionerTests.swift new file mode 100644 index 000000000..35acf3165 --- /dev/null +++ b/CotabbyTests/Services/Runtime/MLX/MLXToolchainProvisionerTests.swift @@ -0,0 +1,277 @@ +import Combine +import CryptoKit +import XCTest +@testable import Cotabby + +@MainActor +final class MLXToolchainProvisionerTests: XCTestCase { + private static var retained: [AnyObject] = [] + private var root: URL! + private var home: URL! + private var paths: MLXPaths! + + override func setUpWithError() throws { + root = FileManager.default.temporaryDirectory.appendingPathComponent("mlx-prov-\(UUID().uuidString)") + home = root.appendingPathComponent("home") + paths = MLXPaths(root: root.appendingPathComponent("Support/Cotabby/MLX")) + try FileManager.default.createDirectory(at: home, withIntermediateDirectories: true) + } + + override func tearDownWithError() throws { + try? FileManager.default.removeItem(at: root) + } + + /// Writes an executable launcher script whose shebang names `python`. + private func script(at url: URL, python: String) throws { + try FileManager.default.createDirectory(at: url.deletingLastPathComponent(), withIntermediateDirectories: true) + try "#!\(python)\nimport mlx_lm\n".write(to: url, atomically: true, encoding: .utf8) + try FileManager.default.setAttributes([.posixPermissions: 0o755], ofItemAtPath: url.path) + } + + private func makeProvisioner( + versions: [String: String], + installer: MLXToolchainInstalling = NeverInstaller() + ) -> MLXToolchainProvisioner { + let provisioner = MLXToolchainProvisioner( + paths: paths, home: home, fileManager: .default, + probeVersion: { python in versions[python.path] }, + installer: installer + ) + Self.retained.append(provisioner) + return provisioner + } + + func test_ownInstallWinsOverExternal() async throws { + try script(at: paths.ownServerExecutable, python: "/own/python") + try script(at: home.appendingPathComponent(".local/bin/mlx_lm.server"), python: "/uv/python") + let state = await makeProvisioner(versions: ["/own/python": "0.32.0", "/uv/python": "0.33.0"]).locate() + XCTAssertEqual(state.toolchain?.pythonExecutable.path, "/own/python") + XCTAssertEqual(state.toolchain?.isCotabbyInstall, true) + } + + /// A rescan while MLX runs must not flash `.checking`: that would turn the server desire into + /// "stopped" and kill the live server. + func test_refreshFromReadyNeverPublishesChecking() async throws { + try script(at: paths.ownServerExecutable, python: "/own/python") + let provisioner = makeProvisioner(versions: ["/own/python": "0.32.0"]) + await provisioner.refresh() + XCTAssertNotNil(provisioner.state.toolchain) + var seen: [MLXToolchainState] = [] + let cancellable = provisioner.$state.sink { seen.append($0) } + await provisioner.refresh() + cancellable.cancel() + XCTAssertFalse(seen.contains(.checking), "saw \(seen)") + XCTAssertNotNil(provisioner.state.toolchain) + } + + func test_externalUvInstallIsUsedWhenNoOwnInstall() async throws { + try script(at: home.appendingPathComponent(".local/bin/mlx_lm.server"), python: "/uv/python") + let state = await makeProvisioner(versions: ["/uv/python": "0.32.0"]).locate() + XCTAssertEqual(state.toolchain?.version, "0.32.0") + XCTAssertEqual(state.toolchain?.isCotabbyInstall, false) + } + + func test_tooOldExternalIsReported() async throws { + try script(at: home.appendingPathComponent(".local/bin/mlx_lm.server"), python: "/uv/python") + let state = await makeProvisioner(versions: ["/uv/python": "0.29.1"]).locate() + XCTAssertEqual(state, .tooOld(version: "0.29.1", path: home.appendingPathComponent(".local/bin/mlx_lm.server").path)) + } + + /// Review focus 5: a half-written install (Cotabby quit mid-install) must not count as installed. + func test_ownInstallWithoutWorkingPythonIsNotInstalled() async throws { + try script(at: paths.ownServerExecutable, python: "/own/python") + let state = await makeProvisioner(versions: [:]).locate() + XCTAssertEqual(state, .notInstalled) + } + + func test_concurrentInstallRequestsShareOneInstall() async throws { + let installer = GatedInstaller() + let provisioner = makeProvisioner(versions: ["/own/python": "0.32.0"], installer: installer) + provisioner.install() + provisioner.install() + await waitUntil { installer.calls == 1 && provisioner.state == .installing(.installingMLX) } + try script(at: paths.ownServerExecutable, python: "/own/python") + installer.release() + await waitUntil { provisioner.state.toolchain != nil } + XCTAssertEqual(installer.calls, 1) + } + + func test_installFailureKeepsTheLogTail() async { + let provisioner = makeProvisioner(versions: [:], installer: FailingInstaller()) + provisioner.install() + await waitUntil { if case .failed = provisioner.state { return true } else { return false } } + XCTAssertEqual(provisioner.state, .failed(reason: MLXInstallError.installFailed(tail: ["boom"]).localizedDescription, logTail: ["boom"])) + } + + /// Remove must not delete the folder under a live uv, and the slot must stay taken until that + /// install has really ended. + func test_removeDuringInstallJoinsTheCancelledInstallBeforeDeleting() async throws { + let installer = GatedInstaller() + let provisioner = makeProvisioner(versions: ["/own/python": "0.32.0"], installer: installer) + provisioner.install() + await waitUntil { installer.calls == 1 } + + let removing = Task { await provisioner.remove() } + // The test has not suspended since `install()`, so `remove()` has not started yet and the + // running install still holds the slot. This second call must be a no-op. + provisioner.install() + XCTAssertEqual(installer.calls, 1, "a second install must not start while the first holds the slot") + + await removing.value + XCTAssertEqual(installer.finished, 1, "remove must wait for the cancelled install to end") + XCTAssertFalse(FileManager.default.fileExists(atPath: paths.root.path)) + + try script(at: paths.ownServerExecutable, python: "/own/python") + provisioner.install() + await waitUntil { installer.calls == 2 } + XCTAssertEqual(installer.finished, 1, "the second install starts only after the first finished") + installer.release() + await waitUntil { provisioner.state.toolchain != nil } + XCTAssertEqual(installer.calls, 2) + XCTAssertEqual(installer.overlaps, 0) + } + + func test_cancelDuringDownloadRevertsInsteadOfFailing() async { + let installer = MLXToolchainInstaller( + paths: paths, + download: { _ in + _ = try? await Task.sleep(for: .seconds(10)) + throw URLError(.cancelled) + }, + runProcess: { _, _, _, _ in + XCTFail("nothing may run after a cancelled download") + return MLXProcessResult(status: 0, stdout: "", stderrTail: []) + }, + verifySignature: { _ in true } + ) + let provisioner = makeProvisioner(versions: [:], installer: installer) + provisioner.install() + await waitUntil { provisioner.state == .installing(.downloadingUV) } + provisioner.cancelInstall() + await waitUntil { provisioner.state == .notInstalled } + XCTAssertEqual(provisioner.state, .notInstalled) + } + + func test_removeDeletesOnlyCotabbysFolder() async throws { + try script(at: paths.ownServerExecutable, python: "/own/python") + let external = home.appendingPathComponent(".local/bin/mlx_lm.server") + try script(at: external, python: "/uv/python") + let provisioner = makeProvisioner(versions: ["/own/python": "0.32.0", "/uv/python": "0.32.0"]) + await provisioner.remove() + XCTAssertFalse(FileManager.default.fileExists(atPath: paths.root.path)) + XCTAssertTrue(FileManager.default.fileExists(atPath: external.path)) + XCTAssertEqual(provisioner.state.toolchain?.isCotabbyInstall, false) + } + + // MARK: - Installer + + func test_installerRejectsAChecksumMismatchBeforeRunningAnything() async { + var ran = false + let installer = MLXToolchainInstaller( + paths: paths, + download: { _ in Data("not uv".utf8) }, + runProcess: { _, _, _, _ in ran = true; return MLXProcessResult(status: 0, stdout: "", stderrTail: []) }, + verifySignature: { _ in true } + ) + do { + try await installer.install(phase: { _ in }) + XCTFail("expected a checksum failure") + } catch { + XCTAssertEqual(error as? MLXInstallError, .checksumMismatch) + } + XCTAssertFalse(ran) + } + + /// The Security check is synchronous and can be slow, so it runs in a detached task, off the + /// main thread, as it does in the installer. + func test_signatureCheckRejectsAnUnrelatedBinary() async { + let signedByAstral = await Task.detached { + MLXToolchainInstaller.isSignedByAstral(URL(fileURLWithPath: "/usr/bin/true")) + }.value + XCTAssertFalse(signedByAstral) + } + + func test_installerRejectsAnUnsignedUV() async { + let archive = Data("pretend archive".utf8) + var commands: [String] = [] + let installer = MLXToolchainInstaller( + paths: paths, + expectedSHA256: SHA256.hash(data: archive).map { String(format: "%02x", $0) }.joined(), + download: { _ in archive }, + runProcess: { executable, _, _, _ in + commands.append(executable.lastPathComponent) + return MLXProcessResult(status: 0, stdout: "", stderrTail: []) + }, + verifySignature: { _ in false } + ) + do { + try await installer.install(phase: { _ in }) + XCTFail("expected a signature failure") + } catch { + XCTAssertEqual(error as? MLXInstallError, .signatureInvalid) + } + XCTAssertEqual(commands, ["tar"], "uv must never run when its signature is not Astral's") + } + + func test_installerRunsUvWithThePinnedCommand() async throws { + let archive = Data("pretend archive".utf8) + var calls: [(String, [String], [String: String])] = [] + var phases: [MLXInstallPhase] = [] + let installer = MLXToolchainInstaller( + paths: paths, + expectedSHA256: SHA256.hash(data: archive).map { String(format: "%02x", $0) }.joined(), + download: { _ in archive }, + runProcess: { executable, arguments, environment, _ in + calls.append((executable.path, arguments, environment)) + return MLXProcessResult(status: 0, stdout: "", stderrTail: []) + }, + verifySignature: { _ in true } + ) + try await installer.install(phase: { phases.append($0) }) + XCTAssertEqual(phases, [.downloadingUV, .verifyingUV, .installingMLX]) + XCTAssertEqual(calls.map(\.0), ["/usr/bin/tar", paths.uvExecutable.path]) + XCTAssertEqual(calls[1].1, ["tool", "install", "mlx-lm==0.32.0", "--python", "3.12"]) + XCTAssertEqual(calls[1].2["UV_TOOL_BIN_DIR"], paths.binDirectory.path) + } +} + +nonisolated private struct NeverInstaller: MLXToolchainInstalling { + func install(phase: @escaping @MainActor (MLXInstallPhase) -> Void) async throws { + XCTFail("install should not run") + } +} + +private struct FailingInstaller: MLXToolchainInstalling { + func install(phase: @escaping @MainActor (MLXInstallPhase) -> Void) async throws { + throw MLXInstallError.installFailed(tail: ["boom"]) + } +} + +/// Holds inside `install` until the test calls `release()` or the install is cancelled. It polls +/// instead of parking on a continuation, so cancellation ends it at once: `Task.sleep` throws +/// `CancellationError` when the install task is cancelled. +@MainActor +private final class GatedInstaller: MLXToolchainInstalling { + private(set) var calls = 0 + private(set) var finished = 0 + /// Calls that began while another was still running. Single-flight keeps this at 0. + private(set) var overlaps = 0 + private var active = 0 + private var released = false + + func install(phase: @escaping @MainActor (MLXInstallPhase) -> Void) async throws { + calls += 1 + if active > 0 { overlaps += 1 } + active += 1 + defer { + active -= 1 + finished += 1 + } + phase(.installingMLX) + while !released { try await Task.sleep(for: .milliseconds(5)) } + } + + func release() { + released = true + } +} diff --git a/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClientTests.swift b/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClientTests.swift index 4700747ae..4fe679710 100644 --- a/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClientTests.swift +++ b/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleAPIClientTests.swift @@ -236,6 +236,8 @@ final class OpenAICompatibleAPIClientTests: XCTestCase { XCTAssertEqual(json["top_p"] as? Double, 0.8) XCTAssertNil(json["messages"]) XCTAssertNil(json["reasoning_effort"], "reasoning_effort belongs to the chat surface only") + XCTAssertNil(json["top_k"]) + XCTAssertNil(json["stop"]) return Self.response( request: request, contentType: "text/event-stream", @@ -258,6 +260,77 @@ final class OpenAICompatibleAPIClientTests: XCTestCase { XCTAssertEqual(partials, [" hel", " hello"]) } + func test_completionGeneration_encodesMLXFieldsOnlyWhenPresent() async throws { + let client = makeClient() + EndpointStubURLProtocol.handler = { request in + let json = try Self.jsonBody(request) + XCTAssertEqual(json["top_k"] as? Int, 20) + XCTAssertEqual(json["min_p"] as? Double, 0.08) + XCTAssertEqual(json["repetition_penalty"] as? Double, 1.05) + XCTAssertEqual(json["seed"] as? Int, 42) + XCTAssertEqual(json["stop"] as? [String], ["\n"]) + return Self.response(request: request, contentType: "text/event-stream", + body: "data: {\"choices\":[{\"text\":\" ok\"}]}\n\ndata: [DONE]\n\n") + } + _ = try await client.generate( + configuration: configuration(mode: .completions), + apiKey: nil, + prompt: "P", + options: .init( + maxPredictionTokens: 8, temperature: 0.1, topP: 0.7, + mlx: MLXCompletionOptions(topK: 20, minP: 0.08, repetitionPenalty: 1.05, seed: 42, stop: ["\n"]) + ), + onPartialRawText: nil + ) + } + + func test_completionGeneration_omitsEmptyStopList() async throws { + let client = makeClient() + EndpointStubURLProtocol.handler = { request in + let json = try Self.jsonBody(request) + XCTAssertEqual(json["top_k"] as? Int, 20) + XCTAssertNil(json["stop"], "multi-line requests must not send an empty stop list") + XCTAssertNil(json["seed"]) + return Self.response(request: request, contentType: "text/event-stream", + body: "data: {\"choices\":[{\"text\":\" ok\"}]}\n\ndata: [DONE]\n\n") + } + _ = try await client.generate( + configuration: configuration(mode: .completions), + apiKey: nil, + prompt: "P", + options: .init( + maxPredictionTokens: 8, temperature: 0.1, topP: 0.7, + mlx: MLXCompletionOptions(topK: 20, minP: 0.08, repetitionPenalty: 1.05, seed: nil, stop: []) + ), + onPartialRawText: nil + ) + } + + func test_generation_stopWhenEndsTheStreamEarly() async throws { + let client = makeClient() + EndpointStubURLProtocol.handler = { request in + Self.response(request: request, contentType: "text/event-stream", body: + "data: {\"choices\":[{\"text\":\" one\"}]}\n\n" + + "data: {\"choices\":[{\"text\":\" two\"}]}\n\n" + + "data: {\"choices\":[{\"text\":\" three\"}]}\n\n" + + "data: [DONE]\n\n") + } + var seenCounts: [Int] = [] + let output = try await client.generate( + configuration: configuration(mode: .completions), + apiKey: nil, + prompt: "P", + options: Self.options, + onPartialRawText: nil, + stopWhen: { accumulated, events in + seenCounts.append(events) + return accumulated.hasSuffix("two") + } + ) + XCTAssertEqual(output, " one two") + XCTAssertEqual(seenCounts, [1, 2]) + } + func test_defaultOllamaPreload_usesNativeRouteLongTimeoutAndKeepsModelResident() async throws { let client = makeClient() EndpointStubURLProtocol.handler = { request in diff --git a/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngineTests.swift b/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngineTests.swift index 307d04762..b848e4f08 100644 --- a/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngineTests.swift +++ b/CotabbyTests/Services/Runtime/OpenAICompatible/OpenAICompatibleSuggestionEngineTests.swift @@ -27,6 +27,7 @@ final class OpenAICompatibleSuggestionEngineTests: XCTestCase { XCTAssertEqual(json["max_tokens"] as? Int, 8) XCTAssertEqual(json["temperature"] as? Double, 0.1) XCTAssertEqual(json["top_p"] as? Double, 0.7) + XCTAssertNil(json["top_k"]) return Self.sse(request, "data: {\"choices\":[{\"text\":\" world\"}]}\n\ndata: [DONE]\n\n") } let engine = makeEngine(mode: .completions, apiKey: "key-123") @@ -80,13 +81,163 @@ final class OpenAICompatibleSuggestionEngineTests: XCTestCase { XCTAssertEqual(EngineStubURLProtocol.requestedURLs, ["http://127.0.0.1:11434/api/generate"]) } + private func mlx404Engine( + health: MLXServerHealth, reports: @escaping @MainActor (Int, String) -> Void + ) -> OpenAICompatibleSuggestionEngine { + EngineStubURLProtocol.handler = { request in + (HTTPURLResponse(url: request.url!, statusCode: 404, httpVersion: nil, headerFields: nil)!, Data()) + } + return makeEngine( + baseURL: "http://127.0.0.1:50123/v1", mode: .completions, backend: .mlx, + onServerBroken: reports, healthCheck: { _ in health }) + } + + func test_mlxBackend_404WithHealth503ReportsBrokenAndSaysItIsRestarting() async { + var reports: [(Int, String)] = [] + let engine = mlx404Engine(health: .generatorDead) { reports.append(($0, $1)) } + await assertThrows(engine, .unavailable("MLX is restarting…")) + XCTAssertEqual(reports.count, 1) + XCTAssertEqual(reports.first?.0, 50123) + XCTAssertEqual(reports.first?.1, "HTTP 404") + } + + func test_mlxBackend_404WithHealthUnreachableReportsBroken() async { + var reports = 0 + let engine = mlx404Engine(health: .unreachable) { _, _ in reports += 1 } + await assertThrows(engine, .unavailable("MLX is restarting…")) + XCTAssertEqual(reports, 1) + } + + func test_mlxBackend_404WithHealthyServerKeepsItAndFailsOnlyThisRequest() async { + var reports = 0 + let engine = mlx404Engine(health: .healthy) { _, _ in reports += 1 } + await assertThrows(engine, .generationFailed("The endpoint returned HTTP 404.")) + XCTAssertEqual(reports, 0) + } + + func test_mlxBackend_cancelledDuringTheHealthProbeNeverReportsBroken() async { + let started = HealthProbeGate() + var reports = 0 + EngineStubURLProtocol.handler = { request in + (HTTPURLResponse(url: request.url!, statusCode: 404, httpVersion: nil, headerFields: nil)!, Data()) + } + let probing = makeEngine( + baseURL: "http://127.0.0.1:50123/v1", mode: .completions, backend: .mlx, + onServerBroken: { _, _ in reports += 1 }, + healthCheck: { _ in + started.open() + // Returns early when cancelled, and then claims "unreachable" like a cancelled URLSession call. + try? await Task.sleep(for: .seconds(10)) + return .unreachable + }) + let task = Task { try await probing.generateSuggestion(for: CotabbyTestFixtures.suggestionRequest()) } + await waitUntil { started.isOpen } + task.cancel() + do { + _ = try await task.value + XCTFail("expected cancellation") + } catch { + guard case .cancelled? = error as? SuggestionClientError else { return XCTFail("\(error)") } + } + XCTAssertEqual(reports, 0) + } + + func test_mlxBackend_streamedGenerationThreadDiedReportsBroken() async { + EngineStubURLProtocol.handler = { request in + Self.sse(request, "data: {\"error\":{\"message\":\"generation thread died\"}}\n\n") + } + var reports: [(Int, String)] = [] + let engine = makeEngine( + baseURL: "http://127.0.0.1:50123/v1", mode: .completions, backend: .mlx, + onServerBroken: { reports.append(($0, $1)) }) + await assertThrows(engine, .unavailable("MLX is restarting…")) + XCTAssertEqual(reports.count, 1) + } + + func test_endpointBackend_404DoesNotReportBroken() async { + EngineStubURLProtocol.handler = { request in + (HTTPURLResponse(url: request.url!, statusCode: 404, httpVersion: nil, headerFields: nil)!, Data()) + } + var reports = 0 + let engine = makeEngine(mode: .completions, onServerBroken: { _, _ in reports += 1 }) + await assertThrows(engine, .generationFailed("The endpoint returned HTTP 404.")) + XCTAssertEqual(reports, 0) + } + // MARK: - Helpers + func test_mlxBackend_sendsMLXSamplingAndSingleLineStop() async throws { + EngineStubURLProtocol.handler = { request in + let json = try Self.jsonBody(request) + XCTAssertEqual(json["top_k"] as? Int, 20) + XCTAssertEqual(json["min_p"] as? Double, 0.08) + XCTAssertEqual(json["repetition_penalty"] as? Double, 1.05) + XCTAssertEqual(json["seed"] as? Int, 42) + XCTAssertEqual(json["stop"] as? [String], ["\n"]) + return Self.sse(request, "data: {\"choices\":[{\"text\":\" world\"}]}\n\ndata: [DONE]\n\n") + } + let engine = makeEngine(mode: .completions, backend: .mlx) + let result = try await engine.generateSuggestion(for: CotabbyTestFixtures.suggestionRequest()) + XCTAssertEqual(result.rawText, " world") + } + + func test_mlxBackend_multiLineSendsNoStop() async throws { + EngineStubURLProtocol.handler = { request in + XCTAssertNil(try Self.jsonBody(request)["stop"]) + return Self.sse(request, "data: {\"choices\":[{\"text\":\" world\"}]}\n\ndata: [DONE]\n\n") + } + let engine = makeEngine(mode: .completions, backend: .mlx) + _ = try await engine.generateSuggestion( + for: CotabbyTestFixtures.suggestionRequest(isMultiLineEnabled: true) + ) + } + + /// The llama runtime stops decoding at a sentence end; MLX gets the same rule by closing the + /// stream, which also makes mlx_lm.server stop on its next token write. + func test_mlxBackend_stopsAtTheFirstSentenceEnd() async throws { + EngineStubURLProtocol.handler = { request in + Self.sse(request, + "data: {\"choices\":[{\"text\":\" Thanks\"}]}\n\n" + + "data: {\"choices\":[{\"text\":\" for the report.\"}]}\n\n" + + "data: {\"choices\":[{\"text\":\" Next week\"}]}\n\n" + + "data: [DONE]\n\n") + } + let engine = makeEngine(mode: .completions, backend: .mlx) + let result = try await engine.generateSuggestion(for: CotabbyTestFixtures.suggestionRequest()) + XCTAssertEqual(result.rawText, " Thanks for the report.") + } + + func test_endpointBackend_doesNotStopEarly() async throws { + EngineStubURLProtocol.handler = { request in + Self.sse(request, + "data: {\"choices\":[{\"text\":\" Thanks for the report.\"}]}\n\n" + + "data: {\"choices\":[{\"text\":\" Next week\"}]}\n\n" + + "data: [DONE]\n\n") + } + let result = try await makeEngine(mode: .completions) + .generateSuggestion(for: CotabbyTestFixtures.suggestionRequest()) + XCTAssertEqual(result.rawText, " Thanks for the report. Next week") + } + + func test_mlxNotReady_throwsUnavailableWithoutNetwork() async { + let engine = makeEngine( + configuration: { throw SuggestionClientError.unavailable("MLX is starting the model…") }, + backend: .mlx + ) + await assertThrows(engine, .unavailable("MLX is starting the model…")) + XCTAssertTrue(EngineStubURLProtocol.requestedURLs.isEmpty) + } + + /// Covers the MLX backend too: it reuses this engine and only changes the sampling fields and + /// the early stop, so the same stub server answers both. private func makeEngine( baseURL: String = OpenAICompatibleEndpointConfiguration.defaultBaseURLString, modelName: String = "local-model", mode: OpenAICompatibleAPIMode = .completions, - apiKey: String? = nil + apiKey: String? = nil, + backend: OpenAICompatibleSuggestionEngine.Backend = .endpoint, + onServerBroken: (@MainActor (Int, String) -> Void)? = nil, + healthCheck: @escaping MLXHealthCheck = { _ in .generatorDead } ) -> OpenAICompatibleSuggestionEngine { makeEngine( configuration: { @@ -96,13 +247,19 @@ final class OpenAICompatibleSuggestionEngineTests: XCTestCase { apiMode: mode ) }, - apiKey: apiKey + apiKey: apiKey, + backend: backend, + onServerBroken: onServerBroken, + healthCheck: healthCheck ) } private func makeEngine( configuration: @escaping @MainActor () throws -> OpenAICompatibleEndpointConfiguration, - apiKey: String? = nil + apiKey: String? = nil, + backend: OpenAICompatibleSuggestionEngine.Backend = .endpoint, + onServerBroken: (@MainActor (Int, String) -> Void)? = nil, + healthCheck: @escaping MLXHealthCheck = { _ in .generatorDead } ) -> OpenAICompatibleSuggestionEngine { let sessionConfiguration = URLSessionConfiguration.ephemeral sessionConfiguration.protocolClasses = [EngineStubURLProtocol.self] @@ -110,7 +267,10 @@ final class OpenAICompatibleSuggestionEngineTests: XCTestCase { let engine = OpenAICompatibleSuggestionEngine( client: client, configurationProvider: configuration, - apiKeyProvider: { apiKey } + apiKeyProvider: { apiKey }, + backend: backend, + onServerBroken: onServerBroken, + healthCheck: healthCheck ) Self.retained.append(contentsOf: [client, engine] as [AnyObject]) return engine @@ -192,3 +352,11 @@ private final class EngineStubURLProtocol: URLProtocol { override func stopLoading() {} } + +/// A flag a `@Sendable` health closure can raise and the test can poll. +final class HealthProbeGate: @unchecked Sendable { + private let lock = NSLock() + private var opened = false + func open() { lock.withLock { opened = true } } + var isOpen: Bool { lock.withLock { opened } } +} diff --git a/CotabbyTests/Services/Runtime/SuggestionEngineRouterTests.swift b/CotabbyTests/Services/Runtime/SuggestionEngineRouterTests.swift index e34c054dc..91a05e20b 100644 --- a/CotabbyTests/Services/Runtime/SuggestionEngineRouterTests.swift +++ b/CotabbyTests/Services/Runtime/SuggestionEngineRouterTests.swift @@ -21,6 +21,7 @@ final class SuggestionEngineRouterRoutingTests: XCTestCase { let endpoint: ScriptedEngine let metrics: PerformanceMetricsStore let quality: SuggestionQualityMetricsStore + let mlx: ScriptedEngine } @MainActor @@ -80,6 +81,7 @@ final class SuggestionEngineRouterRoutingTests: XCTestCase { let llama = ScriptedEngine() let endpoint = ScriptedEngine() let quality = SuggestionQualityMetricsStore(userDefaults: defaults) + let mlx = ScriptedEngine() let router = SuggestionEngineRouter( suggestionSettings: settings, foundationModelEngine: foundation, @@ -88,7 +90,9 @@ final class SuggestionEngineRouterRoutingTests: XCTestCase { qualityMetricsStore: quality, llamaModelNameProvider: { llamaModelName }, openAICompatibleEngine: endpoint, - endpointModelNameProvider: { "endpoint-model" } + endpointModelNameProvider: { "endpoint-model" }, + mlxEngine: mlx, + mlxModelNameProvider: { "mlx-community/m" } ) Self.retained.append(contentsOf: [router, settings, metrics, quality] as [AnyObject]) return Rig( @@ -98,10 +102,52 @@ final class SuggestionEngineRouterRoutingTests: XCTestCase { llama: llama, endpoint: endpoint, metrics: metrics, - quality: quality + quality: quality, + mlx: mlx ) } + /// MLX is a loopback server Cotabby runs itself: requests carrying typing history go to it + /// unchanged (unlike the endpoint), and its latency is recorded under the repo id. + func test_mlxRoutesToMLXEngineAndKeepsHistory() async throws { + let rig = makeRig(engine: .mlx) + let request = CotabbyTestFixtures.suggestionRequest(historyExamples: ["Best regards, Alex"]) + + let result = try await rig.router.generateSuggestion(for: request) + + XCTAssertEqual(rig.mlx.requests.count, 1) + XCTAssertEqual(rig.mlx.requests.first?.historyExamples, ["Best regards, Alex"]) + XCTAssertTrue(rig.endpoint.requests.isEmpty) + XCTAssertTrue(rig.llama.requests.isEmpty) + XCTAssertNil(result.suppressionReason) + XCTAssertEqual(rig.metrics.entries.first?.modelName, "mlx-community/m") + } + + func test_mlxWithoutAnEngineIsUnavailableNotFallback() async { + let defaults = makeDefaults() + let settings = SuggestionSettingsModel(configuration: .standard, userDefaults: defaults) + settings.selectEngine(.mlx) + let llama = ScriptedEngine() + let router = SuggestionEngineRouter( + suggestionSettings: settings, + foundationModelEngine: ScriptedEngine(), + llamaEngine: llama, + performanceMetricsStore: PerformanceMetricsStore(userDefaults: defaults), + qualityMetricsStore: SuggestionQualityMetricsStore(userDefaults: defaults), + llamaModelNameProvider: { nil } + ) + Self.retained.append(contentsOf: [router, settings] as [AnyObject]) + do { + _ = try await router.generateSuggestion(for: CotabbyTestFixtures.suggestionRequest()) + XCTFail("Expected unavailable") + } catch SuggestionClientError.unavailable(let message) { + XCTAssertEqual(message, "Set up MLX in Settings → Engine & Model.") + } catch { + XCTFail("Unexpected \(error)") + } + XCTAssertTrue(llama.requests.isEmpty, "MLX never falls back to another engine") + } + func test_appleIntelligenceSelection_routesToFoundationEngineAndRecordsMetric() async throws { let rig = makeRig(engine: .appleIntelligence) diff --git a/CotabbyTests/Support/Runtime/Hardware/GPUStatisticsReaderTests.swift b/CotabbyTests/Support/Runtime/Hardware/GPUStatisticsReaderTests.swift index d3396832d..cd88ad2e4 100644 --- a/CotabbyTests/Support/Runtime/Hardware/GPUStatisticsReaderTests.swift +++ b/CotabbyTests/Support/Runtime/Hardware/GPUStatisticsReaderTests.swift @@ -75,3 +75,35 @@ final class GPUStatisticsReaderTests: XCTestCase { ) } } + +final class GPUMultiPidTests: XCTestCase { + func testCreatorMatchingIsExactOnPidAndNotPrefixOfLongerPid() { + XCTAssertTrue(GPUStatisticsReader.creatorMatches("pid 123, Cotabby", pids: [5, 123])) + XCTAssertFalse(GPUStatisticsReader.creatorMatches("pid 1234, Other", pids: [123])) + XCTAssertFalse(GPUStatisticsReader.creatorMatches("pid 123, X", pids: [])) + } + + func testTrackerNeedsTwoReadingsThenReportsGrowthOverWallTime() { + var tracker = GPUShareTracker() + let start = Date(timeIntervalSince1970: 1000) + XCTAssertNil(tracker.update(pids: [1, 2], nanoseconds: 0, at: start)) + let share = tracker.update(pids: [2, 1], nanoseconds: 1_000_000_000, at: start.addingTimeInterval(2)) + XCTAssertEqual(try XCTUnwrap(share), 50, accuracy: 0.001) + } + + func testTrackerRestartsWhenPidSetChanges() { + var tracker = GPUShareTracker() + let start = Date(timeIntervalSince1970: 1000) + _ = tracker.update(pids: [1], nanoseconds: 100, at: start) + XCTAssertNil(tracker.update(pids: [1, 2], nanoseconds: 9_000_000_000, at: start.addingTimeInterval(2))) + XCTAssertNotNil(tracker.update(pids: [1, 2], nanoseconds: 9_500_000_000, at: start.addingTimeInterval(4))) + } + + func testTrackerNeverReportsNegativeOrAbove100() { + var tracker = GPUShareTracker() + let start = Date(timeIntervalSince1970: 1000) + _ = tracker.update(pids: [1], nanoseconds: 5_000_000_000, at: start) + XCTAssertNil(tracker.update(pids: [1], nanoseconds: 1, at: start.addingTimeInterval(1))) + XCTAssertEqual(tracker.update(pids: [1], nanoseconds: 9_000_000_000, at: start.addingTimeInterval(2)), 100) + } +} diff --git a/CotabbyTests/Support/Runtime/Hardware/ResourceUsageSummaryTests.swift b/CotabbyTests/Support/Runtime/Hardware/ResourceUsageSummaryTests.swift new file mode 100644 index 000000000..38b8ff767 --- /dev/null +++ b/CotabbyTests/Support/Runtime/Hardware/ResourceUsageSummaryTests.swift @@ -0,0 +1,44 @@ +import XCTest +@testable import Cotabby + +/// Formatting of the menu bar's "Cotabby uses ..." line, plus the sampler's independence from the +/// shared CPU baseline. +final class ResourceUsageSummaryTests: XCTestCase { + private func summary( + cpu: Double = 34.2, bytes: UInt64 = 2_684_354_560, gpu: Double? = 12.4, mlx: Bool = false + ) -> ResourceUsageSummary { + ResourceUsageSummary(cpuPercent: cpu, memoryBytes: bytes, gpuPercent: gpu, includesMLXServer: mlx) + } + + func testFullLine() { + XCTAssertEqual( + ResourceUsageSummary.menuLine(summary()), "Cotabby uses CPU 34% · RAM 2.5 GB · GPU 12%") + } + + func testMLXSuffix() { + XCTAssertTrue(ResourceUsageSummary.menuLine(summary(mlx: true)).hasSuffix("GPU 12% incl. MLX")) + } + + func testUnknownGPUShowsDash() { + XCTAssertTrue(ResourceUsageSummary.menuLine(summary(gpu: nil)).hasSuffix("GPU —")) + } + + func testCPURoundsToWholePercentAndMayExceed100() { + XCTAssertTrue(ResourceUsageSummary.menuLine(summary(cpu: 229.6)).contains("CPU 230%")) + XCTAssertTrue(ResourceUsageSummary.menuLine(summary(cpu: 0.2)).contains("CPU 0%")) + } + + func testMemoryUnits() { + XCTAssertEqual(ResourceUsageSummary.memoryText(150 * 1_048_576), "150 MB") + XCTAssertEqual(ResourceUsageSummary.memoryText(1_073_741_824), "1.0 GB") + XCTAssertEqual(ResourceUsageSummary.memoryText(1_073_741_824 - 1), "1.0 GB", "rounds up to GB, never 1024 MB") + XCTAssertEqual(ResourceUsageSummary.memoryText(0), "0 MB") + } + + func testMenuSamplerDoesNotTouchTheSharedCPUBaseline() { + let before = SystemResourceSampler.sharedBaseline.withLock { $0.count } + let sampler = MenuResourceSampler() + _ = sampler.sample(extraProcessID: getpid()) + XCTAssertEqual(SystemResourceSampler.sharedBaseline.withLock { $0.count }, before) + } +} diff --git a/CotabbyTests/Support/Runtime/Hardware/SystemResourceSamplerTests.swift b/CotabbyTests/Support/Runtime/Hardware/SystemResourceSamplerTests.swift index fa28a1202..c0d4a0dd3 100644 --- a/CotabbyTests/Support/Runtime/Hardware/SystemResourceSamplerTests.swift +++ b/CotabbyTests/Support/Runtime/Hardware/SystemResourceSamplerTests.swift @@ -37,6 +37,27 @@ final class SystemResourceSamplerTests: XCTestCase { XCTAssertGreaterThanOrEqual(second.cpuPercent, 0) } + func test_cpuPercent_convertsMachTicksWithTheTimebase() { + // 125/3 is Apple Silicon's timebase: 24 ticks per 1000 ns. 24M ticks = 1e9 ns over 2 s = 50%. + let percent = SystemResourceSampler.cpuPercent( + previousTicks: 0, currentTicks: 24_000_000, elapsedSeconds: 2, timebase: (125, 3)) + XCTAssertEqual(percent, 24_000_000.0 * 125 / 3 / 2e9 * 100, accuracy: 0.0001) + XCTAssertEqual(SystemResourceSampler.cpuPercent( + previousTicks: 10, currentTicks: 5, elapsedSeconds: 1, timebase: (1, 1)), 0, "counter went backwards") + XCTAssertEqual(SystemResourceSampler.cpuPercent( + previousTicks: 0, currentTicks: 5, elapsedSeconds: 0, timebase: (1, 1)), 0) + } + + func test_extraProcessIDs_addTheirFootprintAndGoneProcessesAddNothing() { + let own = SystemResourceSampler.sample().footprintBytes + let withSelfAsExtra = SystemResourceSampler.sample(extraProcessIDs: [getpid()]) + XCTAssertGreaterThan(withSelfAsExtra.footprintBytes, own, "the extra pid's footprint is added") + XCTAssertLessThan( + SystemResourceSampler.sample(extraProcessIDs: [Int32.max - 1]).footprintBytes, own * 2, + "a pid that does not exist contributes nothing") + XCTAssertFalse(withSelfAsExtra.includesMLXServer, "the flag is set by the caller, not the sampler") + } + func test_sampleValues_compareByBothFields() { // The sample's synthesized Equatable inherits the app module's default MainActor // isolation, so the comparisons run through the main-actor hop helper. diff --git a/CotabbyTests/Support/Runtime/LocalRuntimeResidencyPolicyTests.swift b/CotabbyTests/Support/Runtime/LocalRuntimeResidencyPolicyTests.swift index 540750641..858d13de1 100644 --- a/CotabbyTests/Support/Runtime/LocalRuntimeResidencyPolicyTests.swift +++ b/CotabbyTests/Support/Runtime/LocalRuntimeResidencyPolicyTests.swift @@ -21,17 +21,19 @@ final class LocalRuntimeResidencyPolicyTests: XCTestCase { } } - func test_endpointNeverKeepsTheModelLoaded() { - for fallback in [false, true] { - for keepLoaded in [false, true] { - XCTAssertFalse( - LocalRuntimeResidencyPolicy.keepsModelLoaded( - engine: .openAICompatible, - isAppleLanguageFallbackEnabled: fallback, - keepsFallbackModelLoaded: keepLoaded - ), - "The endpoint runs its own model, so a resident GGUF would only duplicate memory" - ) + func test_endpointAndMLXNeverKeepTheModelLoaded() { + for engine in [SuggestionEngineKind.openAICompatible, .mlx] { + for fallback in [false, true] { + for keepLoaded in [false, true] { + XCTAssertFalse( + LocalRuntimeResidencyPolicy.keepsModelLoaded( + engine: engine, + isAppleLanguageFallbackEnabled: fallback, + keepsFallbackModelLoaded: keepLoaded + ), + "\(engine) runs its own model, so a resident GGUF would only duplicate memory" + ) + } } } } diff --git a/CotabbyTests/Support/Runtime/MLX/MLXCacheScannerTests.swift b/CotabbyTests/Support/Runtime/MLX/MLXCacheScannerTests.swift new file mode 100644 index 000000000..5f0f12173 --- /dev/null +++ b/CotabbyTests/Support/Runtime/MLX/MLXCacheScannerTests.swift @@ -0,0 +1,197 @@ +import XCTest +@testable import Cotabby + +/// Builds real fixture trees in a temp folder shaped like huggingface_hub's cache +/// (models--o--n/{blobs,refs,snapshots/}) and checks which repos count as usable MLX models. +final class MLXCacheScannerTests: XCTestCase { + private var root: URL! + + override func setUpWithError() throws { + root = FileManager.default.temporaryDirectory + .appendingPathComponent("mlx-scan-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true) + } + + override func tearDownWithError() throws { + try? FileManager.default.removeItem(at: root) + } + + /// Writes one cached repo. `files` maps snapshot file names to contents. + @discardableResult + private func repo( + _ folder: String, + revision: String = "rev1", + refsMain: Bool = true, + files: [String: String], + incompleteBlob: Bool = false + ) throws -> URL { + let repo = root.appendingPathComponent(folder, isDirectory: true) + let snapshot = repo.appendingPathComponent("snapshots/\(revision)", isDirectory: true) + try FileManager.default.createDirectory(at: snapshot, withIntermediateDirectories: true) + try FileManager.default.createDirectory(at: repo.appendingPathComponent("blobs"), withIntermediateDirectories: true) + for (name, contents) in files { + try contents.write(to: snapshot.appendingPathComponent(name), atomically: true, encoding: .utf8) + } + if refsMain { + try FileManager.default.createDirectory(at: repo.appendingPathComponent("refs"), withIntermediateDirectories: true) + try revision.write(to: repo.appendingPathComponent("refs/main"), atomically: true, encoding: .utf8) + } + if incompleteBlob { + try "partial".write(to: repo.appendingPathComponent("blobs/abc.incomplete"), atomically: true, encoding: .utf8) + } + return repo + } + + private func modelFiles(type: String, architectures: String = "[\"X\"]") -> [String: String] { + [ + "config.json": "{\"model_type\":\"\(type)\",\"architectures\":\(architectures)}", + "model.safetensors": "weights", + "tokenizer.json": "{}" + ] + } + + func test_qualifyingModelIsListedWithTypeAndSize() throws { + try repo("models--mlx-community--gemma-4-e4b-5bit", files: modelFiles(type: "gemma4")) + let models = MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]) + XCTAssertEqual(models.map(\.repoID.value), ["mlx-community/gemma-4-e4b-5bit"]) + XCTAssertEqual(models.first?.modelType, "gemma4") + XCTAssertGreaterThan(models.first?.sizeBytes ?? 0, 0) + XCTAssertEqual(models.first?.isInstruct, false) + } + + /// A file still downloading has an `.incomplete` blob and no snapshot link yet, so the snapshot + /// misses it. An unrelated partial blob (another revision, an unused file) must not hide a + /// complete snapshot. + func test_incompleteBlobsMatterOnlyThroughTheSnapshot() throws { + var downloading = modelFiles(type: "gemma4"); downloading["model.safetensors"] = nil + try repo("models--a--partial", files: downloading, incompleteBlob: true) + try repo("models--a--complete", files: modelFiles(type: "gemma4"), incompleteBlob: true) + XCTAssertEqual(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).map(\.repoID.value), + ["a/complete"]) + } + + /// A download stopped between shards leaves the config, the tokenizer and some shards: the + /// index names a shard that is not there, so the model is not downloaded. + func test_shardedModelNeedsEveryShardTheIndexNames() throws { + var files = modelFiles(type: "gemma4"); files["model.safetensors"] = nil + files["model.safetensors.index.json"] = """ + {"weight_map": {"a": "model-00001-of-00002.safetensors", "b": "model-00002-of-00002.safetensors"}} + """ + files["model-00001-of-00002.safetensors"] = "w1" + try repo("models--a--sharded", files: files) + XCTAssertTrue(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).isEmpty) + + files["model-00002-of-00002.safetensors"] = "w2" + try repo("models--a--sharded", files: files) + XCTAssertEqual(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).count, 1) + } + + /// A snapshot link whose blob was deleted cannot be loaded. + func test_danglingSnapshotLinkIsNotDownloaded() throws { + let repoURL = try repo("models--a--dangling", files: modelFiles(type: "gemma4")) + let weights = repoURL.appendingPathComponent("snapshots/rev1/model.safetensors") + try FileManager.default.removeItem(at: weights) + try FileManager.default.createSymbolicLink( + atPath: weights.path, withDestinationPath: "../../blobs/gone") + XCTAssertTrue(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).isEmpty) + } + + func test_missingWeightsOrTokenizerIsSkipped() throws { + var noWeights = modelFiles(type: "gemma4"); noWeights["model.safetensors"] = nil + try repo("models--a--noweights", files: noWeights) + var noTokenizer = modelFiles(type: "gemma4"); noTokenizer["tokenizer.json"] = nil + try repo("models--a--notok", files: noTokenizer) + XCTAssertTrue(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).isEmpty) + } + + func test_embeddingRerankerAndSpeechReposAreSkipped() throws { + try repo("models--mlx-community--Qwen3-Embedding-0.6B-8bit", files: modelFiles(type: "qwen3")) + try repo("models--mlx-community--Qwen3-TTS-12Hz-1.7B-Base-bf16", files: modelFiles(type: "qwen3")) + try repo("models--a--plain", files: modelFiles(type: "qwen3", architectures: "[\"BertForSequenceClassification\"]")) + XCTAssertTrue(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["qwen3"]).isEmpty) + } + + func test_detailedScanReportsRejectedRepos() throws { + try repo("models--mlx-community--Qwen3-Embedding-0.6B-8bit", files: modelFiles(type: "qwen3")) + try repo("models--mlx-community--Qwen3-TTS-12Hz-1.7B-Base-bf16", files: modelFiles(type: "qwen3")) + // Still downloading: the weights blob is `.incomplete` and not linked into the snapshot yet. + var downloading = modelFiles(type: "qwen3"); downloading["model.safetensors"] = nil + try repo("models--a--partial", files: downloading, incompleteBlob: true) + try repo("models--a--ok", files: modelFiles(type: "qwen3")) + let result = MLXCacheScanner.scanDetailed(cacheRoot: root, supportedModelTypes: ["qwen3"]) + XCTAssertEqual(result.models.map(\.repoID.value), ["a/ok"]) + XCTAssertEqual(result.rejected[MLXRepoID("mlx-community/Qwen3-Embedding-0.6B-8bit")!], .embeddingModel) + XCTAssertEqual(result.rejected[MLXRepoID("mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16")!], .speechModel) + XCTAssertNil(result.rejected[MLXRepoID("a/partial")!], "an incomplete download is just not downloaded") + } + + func test_partialSnapshotsAreNeitherListedNorRejected() throws { + try repo("models--a--configonly", files: ["config.json": "{\"model_type\":\"qwen3\"}"]) + var noTokenizer = modelFiles(type: "qwen3"); noTokenizer["tokenizer.json"] = nil + try repo("models--a--notok", files: noTokenizer) + let result = MLXCacheScanner.scanDetailed(cacheRoot: root, supportedModelTypes: ["qwen3"]) + XCTAssertTrue(result.models.isEmpty) + XCTAssertTrue(result.rejected.isEmpty) + XCTAssertEqual( + MLXServerDesireResolver.desire(for: .init( + engine: .mlx, isAppleSilicon: true, + toolchain: .ready(MLXToolchain( + serverExecutable: URL(fileURLWithPath: "/s"), pythonExecutable: URL(fileURLWithPath: "/p"), + version: "0.32.0", isCotabbyInstall: true)), + modelID: "a/configonly", cachedModels: result.models, promptCacheBytes: 1 << 30, + rejectedModels: result.rejected)), + .unavailable(.modelNotDownloaded(MLXRepoID("a/configonly")!))) + } + + func test_unsupportedModelTypeIsSkippedUnlessTheSetIsUnknown() throws { + try repo("models--a--odd", files: modelFiles(type: "whisper_like")) + XCTAssertTrue(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).isEmpty) + // Before mlx-lm is installed the supported set is unknown; models are listed unfiltered. + XCTAssertEqual(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: nil).count, 1) + } + + func test_instructNamesAreMarked() throws { + try repo("models--mlx-community--gemma-4-e4b-it-4bit", files: modelFiles(type: "gemma4")) + try repo("models--mlx-community--Llama-3.2-3B-Instruct-4bit", files: modelFiles(type: "llama")) + let models = MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4", "llama"]) + XCTAssertEqual(models.filter(\.isInstruct).count, 2) + XCTAssertFalse(MLXCacheScanner.isInstructName("gemma-4-e4b-5bit")) + XCTAssertFalse(MLXCacheScanner.isInstructName("unit-test"), "'it' must be a whole token") + } + + /// The server loads the repo id offline, which huggingface_hub resolves through `refs/main`. A + /// snapshot without it (cached by commit) could not be loaded, so it is not downloaded. + func test_onlyTheSnapshotRefsMainNamesCounts() throws { + let folder = "models--a--revs" + try repo(folder, revision: "new", refsMain: false, files: modelFiles(type: "gemma4")) + XCTAssertTrue(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).isEmpty, + "no refs/main: the offline server could not resolve the repo") + + try repo(folder, revision: "old", files: ["config.json": "{}"]) + XCTAssertTrue(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]).isEmpty, + "refs/main points at the snapshot without weights") + + try "new".write(to: root.appendingPathComponent("\(folder)/refs/main"), atomically: true, encoding: .utf8) + XCTAssertEqual(MLXCacheScanner.scan(cacheRoot: root, supportedModelTypes: ["gemma4"]) + .first?.snapshotURL.lastPathComponent, "new") + } + + func test_cacheRootFollowsHuggingFaceEnvironment() { + let home = URL(fileURLWithPath: "/Users/u") + XCTAssertEqual(MLXCacheScanner.cacheRoot(environment: [:], home: home).path, "/Users/u/.cache/huggingface/hub") + XCTAssertEqual(MLXCacheScanner.cacheRoot(environment: ["HF_HOME": "/hf"], home: home).path, "/hf/hub") + XCTAssertEqual(MLXCacheScanner.cacheRoot(environment: ["HF_HOME": "/hf", "HF_HUB_CACHE": "/cache"], home: home).path, "/cache") + } + + func test_downloadProgressCountsIncompleteBlobs() throws { + let id = MLXRepoID("a/b")! + let blobs = root.appendingPathComponent("models--a--b/blobs", isDirectory: true) + try FileManager.default.createDirectory(at: blobs, withIntermediateDirectories: true) + try Data(count: 300).write(to: blobs.appendingPathComponent("x")) + try Data(count: 200).write(to: blobs.appendingPathComponent("y.incomplete")) + XCTAssertEqual(MLXCacheScanner.downloadedBytes(repo: id, cacheRoot: root), 500) + XCTAssertEqual(MLXCacheScanner.progress(downloadedBytes: 500, totalBytes: 1000), 0.5) + XCTAssertEqual(MLXCacheScanner.progress(downloadedBytes: 1500, totalBytes: 1000), 1.0) + XCTAssertNil(MLXCacheScanner.progress(downloadedBytes: 1, totalBytes: 0)) + } +} diff --git a/CotabbyTests/Support/Runtime/MLX/MLXModelCompatibilityTests.swift b/CotabbyTests/Support/Runtime/MLX/MLXModelCompatibilityTests.swift new file mode 100644 index 000000000..5cd3a1062 --- /dev/null +++ b/CotabbyTests/Support/Runtime/MLX/MLXModelCompatibilityTests.swift @@ -0,0 +1,74 @@ +import XCTest +@testable import Cotabby + +final class MLXModelCompatibilityTests: XCTestCase { + private let goodFiles = ["config.json", "model.safetensors", "tokenizer.json"] + + private func verdict( + _ name: String = "gemma-4-e4b-5bit", + files: [String]? = nil, + config: [String: Any]? = ["model_type": "gemma4", "architectures": ["Gemma4ForCausalLM"]], + supported: Set? = ["gemma4"] + ) -> MLXModelCompatibility.Verdict { + MLXModelCompatibility.verdict( + repoName: name, fileNames: files ?? goodFiles, config: config, supportedModelTypes: supported) + } + + func test_baseModelIsCompatible() { + XCTAssertEqual(verdict(), .compatible(modelType: "gemma4", isInstruct: false)) + } + + func test_instructModelIsCompatibleAndMarked() { + XCTAssertEqual(verdict("gemma-4-e4b-it-4bit"), .compatible(modelType: "gemma4", isInstruct: true)) + XCTAssertFalse(MLXModelCompatibility.isInstructName("unit-test"), "'it' must be a whole token") + } + + func test_embeddingByName() { + XCTAssertEqual(verdict("Qwen3-Embedding-0.6B-8bit"), .incompatible(.embeddingModel)) + } + + func test_rerankerByName() { + XCTAssertEqual(verdict("bge-reranker-v2"), .incompatible(.rerankerModel)) + } + + func test_ttsByName() { + XCTAssertEqual(verdict("Qwen3-TTS-12Hz-1.7B-Base-bf16"), .incompatible(.speechModel)) + } + + func test_speechByArchitecture() { + let config: [String: Any] = ["model_type": "gemma4", "architectures": ["Wav2Vec2ForCTC"]] + XCTAssertEqual(verdict(config: config), .incompatible(.speechModel)) + } + + func test_classifierByArchitecture() { + let config: [String: Any] = ["model_type": "gemma4", "architectures": ["BertForSequenceClassification"]] + XCTAssertEqual(verdict(config: config), .incompatible(.classifierModel)) + } + + func test_unsupportedModelType() { + XCTAssertEqual(verdict(config: ["model_type": "xyz"]), .incompatible(.unsupportedArchitecture("xyz"))) + } + + func test_unknownSupportedSetStillRejectsEmbeddingButAcceptsOddType() { + XCTAssertEqual(verdict("a-embed", supported: nil), .incompatible(.embeddingModel)) + XCTAssertEqual(verdict(config: ["model_type": "xyz"], supported: nil), + .compatible(modelType: "xyz", isInstruct: false)) + } + + func test_missingFiles() { + XCTAssertEqual(verdict(files: ["model.safetensors", "tokenizer.json"]), .incompatible(.missingConfig)) + XCTAssertEqual(verdict(files: ["config.json", "tokenizer.json"]), .incompatible(.missingWeights)) + XCTAssertEqual(verdict(files: ["config.json", "model.safetensors"]), .incompatible(.missingTokenizer)) + } + + func test_unreadableConfig() { + XCTAssertEqual(verdict(config: nil), .incompatible(.unreadableConfig)) + XCTAssertEqual(verdict(config: ["architectures": ["X"]]), .incompatible(.unreadableConfig)) + } + + func test_messagesAreOneSentence() { + XCTAssertEqual(MLXModelCompatibility.Reason.embeddingModel.message, "Embedding model: it can't write text.") + XCTAssertEqual(MLXModelCompatibility.Reason.unsupportedArchitecture("xyz").message, + "Architecture “xyz” isn't supported by the installed mlx-lm.") + } +} diff --git a/CotabbyTests/Support/Runtime/MLX/MLXServerDesireResolverTests.swift b/CotabbyTests/Support/Runtime/MLX/MLXServerDesireResolverTests.swift new file mode 100644 index 000000000..b43101dd6 --- /dev/null +++ b/CotabbyTests/Support/Runtime/MLX/MLXServerDesireResolverTests.swift @@ -0,0 +1,84 @@ +import XCTest +@testable import Cotabby + +final class MLXServerDesireResolverTests: XCTestCase { + private let toolchain = MLXToolchain( + serverExecutable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), + pythonExecutable: URL(fileURLWithPath: "/mlx/python"), + version: "0.32.0", + isCotabbyInstall: true + ) + private let model = MLXCachedModel( + repoID: MLXRepoID("mlx-community/gemma-4-e4b-5bit")!, snapshotURL: URL(fileURLWithPath: "/c"), + sizeBytes: 1, modelType: "gemma4", isInstruct: false + ) + + private func desire( + engine: SuggestionEngineKind = .mlx, + appleSilicon: Bool = true, + toolchain: MLXToolchainState? = nil, + modelID: String = "mlx-community/gemma-4-e4b-5bit", + cached: [MLXCachedModel]?? = .none, + rejected: [MLXRepoID: MLXModelCompatibility.Reason] = [:] + ) -> MLXServerDesire { + MLXServerDesireResolver.desire(for: .init( + engine: engine, + isAppleSilicon: appleSilicon, + toolchain: toolchain ?? .ready(self.toolchain), + modelID: modelID, + cachedModels: cached ?? [model], + promptCacheBytes: 1 << 30, + rejectedModels: rejected + )) + } + + func test_selectedModelRejectedByCompatibilityIsIncompatibleNotMissing() { + let id = MLXRepoID("a/embed")! + XCTAssertEqual( + desire(modelID: "a/embed", cached: [], rejected: [id: .embeddingModel]), + .unavailable(.modelIncompatible(id, "Embedding model: it can't write text."))) + XCTAssertEqual(desire(modelID: "a/embed", cached: []), .unavailable(.modelNotDownloaded(id))) + XCTAssertEqual( + MLXUnavailableReason.modelIncompatible(id, "x").message, "a/embed can't run on MLX: x") + } + + func test_otherEnginesStopTheServer() { + XCTAssertEqual(desire(engine: .llamaOpenSource), .stopped) + XCTAssertEqual(desire(engine: .openAICompatible), .stopped) + } + + func test_intelIsUnsupported() { + XCTAssertEqual(desire(appleSilicon: false), .unavailable(.unsupportedHardware)) + } + + func test_toolchainStates() { + XCTAssertEqual(desire(toolchain: .checking), .stopped) + XCTAssertEqual(desire(toolchain: .installing(.installingMLX)), .stopped) + XCTAssertEqual(desire(toolchain: .notInstalled), .unavailable(.notInstalled)) + XCTAssertEqual(desire(toolchain: .tooOld(version: "0.20.0", path: "/x")), .unavailable(.notInstalled)) + XCTAssertEqual(desire(toolchain: .failed(reason: "x", logTail: [])), .unavailable(.notInstalled)) + } + + func test_noOrInvalidModelIsNoModel() { + XCTAssertEqual(desire(modelID: ""), .unavailable(.noModel)) + XCTAssertEqual(desire(modelID: "not a repo"), .unavailable(.noModel)) + } + + func test_cacheNotScannedYetWaits() { + XCTAssertEqual(desire(cached: .some(nil)), .stopped) + } + + /// Review focus 2. + func test_selectedModelMissingFromCache_isModelNotDownloaded() { + XCTAssertEqual(desire(cached: .some([])), + .unavailable(.modelNotDownloaded(MLXRepoID("mlx-community/gemma-4-e4b-5bit")!))) + } + + func test_everythingPresentRunsTheServer() { + XCTAssertEqual(desire(), .running(MLXServerLaunch( + executable: URL(fileURLWithPath: "/mlx/bin/mlx_lm.server"), + modelID: "mlx-community/gemma-4-e4b-5bit", + promptCacheBytes: 1 << 30 + ))) + } +} diff --git a/CotabbyTests/Support/Runtime/MLX/MLXServerLogFilterTests.swift b/CotabbyTests/Support/Runtime/MLX/MLXServerLogFilterTests.swift new file mode 100644 index 000000000..aa2ad9d4c --- /dev/null +++ b/CotabbyTests/Support/Runtime/MLX/MLXServerLogFilterTests.swift @@ -0,0 +1,92 @@ +import XCTest +@testable import Cotabby + +/// Pins the stderr noise filter and the "generator died" classifier with a realistic mlx_lm.server +/// transcript: access-log lines, an early-disconnect BrokenPipe traceback, then the real failure. +final class MLXServerLogFilterTests: XCTestCase { + private let access = #"127.0.0.1 - - [14/Oct/2026 10:00:01] "POST /v1/completions HTTP/1.1" 200 -"# + private let brokenPipe = [ + "----------------------------------------", + "Exception occurred during processing of request from ('127.0.0.1', 51234)", + "Traceback (most recent call last):", + #" File "/python/lib/python3.12/socketserver.py", line 318, in _handle_request_noblock"#, + " self.process_request(request, client_address)", + #" File "/python/lib/python3.12/socketserver.py", line 766, in finish"#, + " self.wfile.flush()", + "BrokenPipeError: [Errno 32] Broken pipe", + "----------------------------------------" + ] + private let reset = [ + "Traceback (most recent call last):", + #" File "/python/lib/python3.12/socketserver.py", line 318, in handle"#, + " self.rfile.readline()", + "ConnectionResetError: [Errno 54] Connection reset by peer" + ] + private let real = [ + "mlx_lm.server generation thread died: Resource limit exceeded", + "Traceback (most recent call last):", + #" File "/mlx_lm/server.py", line 700, in _run_generate"#, + " response = next(gen)", + "RuntimeError: [metal::malloc] Resource limit exceeded" + ] + + func test_dropsAccessLines() { + XCTAssertEqual(MLXServerLogFilter.relevantTail([access, "real line", access], limit: 20), ["real line"]) + } + + func test_dropsWholeBrokenPipeAndResetBlocksButKeepsTheRealFailure() { + let lines = [access] + brokenPipe + [access] + reset + [access] + real + XCTAssertEqual(MLXServerLogFilter.relevantTail(lines, limit: 20), real) + } + + func test_keepsATracebackWithAnotherError() { + let lines = brokenPipe + real + let kept = MLXServerLogFilter.relevantTail(lines, limit: 50) + XCTAssertEqual(kept, real) + XCTAssertTrue(kept.contains("RuntimeError: [metal::malloc] Resource limit exceeded")) + } + + func test_keepsTheLastLimitLinesAfterFiltering() { + let lines = (1...30).map { "line \($0)" } + [access] + let kept = MLXServerLogFilter.relevantTail(lines, limit: 5) + XCTAssertEqual(kept, (26...30).map { "line \($0)" }) + } + + func test_anUnfinishedTracebackIsKept() { + let partial = ["Traceback (most recent call last):", #" File "x.py", line 1, in f"#] + XCTAssertEqual(MLXServerLogFilter.relevantTail(partial, limit: 20), partial) + } + + func test_classifiesGeneratorDeath() { + XCTAssertEqual(MLXGeneratorFailure.reason(for: OpenAICompatibleClientError.server(statusCode: 404)), "HTTP 404") + // mlx_lm never answers 500; only 404 is a suspect. + XCTAssertNil(MLXGeneratorFailure.reason(for: OpenAICompatibleClientError.server(statusCode: 500))) + XCTAssertEqual( + MLXGeneratorFailure.reason(for: OpenAICompatibleClientError.streamError("generation thread died: x")), + "generation thread died") + XCTAssertNil(MLXGeneratorFailure.reason(for: OpenAICompatibleClientError.server(statusCode: 401))) + XCTAssertNil(MLXGeneratorFailure.reason(for: OpenAICompatibleClientError.streamError("model missing"))) + XCTAssertNil(MLXGeneratorFailure.reason(for: URLError(.timedOut))) + } +} + +/// The server log file: header line per start, raw stderr appended, one-step rotation at the cap. +final class MLXServerLogWriterTests: XCTestCase { + func test_rotatesOnceAtTheSizeCapAndWritesHeaders() throws { + let directory = FileManager.default.temporaryDirectory.appendingPathComponent("mlx-log-\(UUID().uuidString)") + defer { try? FileManager.default.removeItem(at: directory) } + let url = directory.appendingPathComponent("mlx-server.log") + let writer = MLXServerLogWriter(fileURL: url, sizeCapBytes: 200) + + writer.writeHeader(pid: 42, model: "org/m", port: 50123, date: Date(timeIntervalSince1970: 0)) + let header = try String(contentsOf: url, encoding: .utf8) + XCTAssertTrue(header.contains("pid=42") && header.contains("model=org/m") && header.contains("port=50123"), header) + + writer.write(Data(String(repeating: "a", count: 250).utf8) + Data("\n".utf8)) + // Over the cap: the next write moves the old file to .1 and starts a fresh one. + writer.write(Data("fresh line\n".utf8)) + let rotated = try String(contentsOf: directory.appendingPathComponent("mlx-server.log.1"), encoding: .utf8) + XCTAssertTrue(rotated.contains("pid=42")) + XCTAssertEqual(try String(contentsOf: url, encoding: .utf8), "fresh line\n") + } +} diff --git a/CotabbyTests/Support/Runtime/MLX/MLXToolchainRulesTests.swift b/CotabbyTests/Support/Runtime/MLX/MLXToolchainRulesTests.swift new file mode 100644 index 000000000..415fc42e6 --- /dev/null +++ b/CotabbyTests/Support/Runtime/MLX/MLXToolchainRulesTests.swift @@ -0,0 +1,101 @@ +import XCTest +@testable import Cotabby + +final class MLXToolchainRulesTests: XCTestCase { + private let paths = MLXPaths(root: URL(fileURLWithPath: "/S/Cotabby/MLX")) + private let home = URL(fileURLWithPath: "/Users/u") + + func test_candidatesAreOwnInstallThenUvThenHomebrew() { + XCTAssertEqual(MLXToolchainRules.serverCandidates(paths: paths, home: home).map(\.path), [ + "/S/Cotabby/MLX/bin/mlx_lm.server", + "/Users/u/.local/bin/mlx_lm.server", + "/opt/homebrew/bin/mlx_lm.server" + ]) + } + + func test_interpreterFromPlainShebang() { + XCTAssertEqual( + MLXToolchainRules.interpreter(fromScript: "#!/Users/u/.local/share/uv/tools/mlx-lm/bin/python\nimport x")?.path, + "/Users/u/.local/share/uv/tools/mlx-lm/bin/python" + ) + } + + /// uv and pip write a /bin/sh trampoline when the Python path has a space, which Cotabby's own + /// install always does ("Application Support"). + func test_interpreterFromShellTrampoline() { + let script = "#!/bin/sh\n'''exec' '/S/Cotabby Dev/MLX/tools/mlx-lm/bin/python' \"$0\" \"$@\"\n' '''\nimport sys\n" + XCTAssertEqual(MLXToolchainRules.interpreter(fromScript: script)?.path, + "/S/Cotabby Dev/MLX/tools/mlx-lm/bin/python") + } + + func test_interpreterRejectsEnvAndGarbage() { + XCTAssertNil(MLXToolchainRules.interpreter(fromScript: "#!/usr/bin/env python3\n")) + XCTAssertNil(MLXToolchainRules.interpreter(fromScript: "print('hi')")) + XCTAssertNil(MLXToolchainRules.interpreter(fromScript: "#!/bin/sh\necho hi\n")) + XCTAssertNil(MLXToolchainRules.interpreter(fromScript: "")) + } + + func test_versionComparison() { + XCTAssertTrue(MLXToolchainRules.isVersion("0.32.0", atLeast: "0.32.0")) + XCTAssertTrue(MLXToolchainRules.isVersion("0.32.1", atLeast: "0.32.0")) + XCTAssertTrue(MLXToolchainRules.isVersion("0.40", atLeast: "0.32.0")) + XCTAssertTrue(MLXToolchainRules.isVersion("1.0.0rc1", atLeast: "0.32.0")) + XCTAssertFalse(MLXToolchainRules.isVersion("0.31.9", atLeast: "0.32.0")) + XCTAssertFalse(MLXToolchainRules.isVersion("0.9.0", atLeast: "0.32.0")) + XCTAssertFalse(MLXToolchainRules.isVersion("garbage", atLeast: "0.32.0")) + } + + func test_serverCommandLineIsExact() { + XCTAssertEqual( + MLXToolchainRules.serverArguments(modelID: "mlx-community/gemma-4-e4b-5bit", port: 50123, promptCacheBytes: 1 << 30), + ["--model", "mlx-community/gemma-4-e4b-5bit", "--host", "127.0.0.1", "--port", "50123", + "--prompt-cache-size", "2", "--prompt-cache-bytes", "1073741824", "--log-level", "WARNING"] + ) + } + + func test_environmentsDropPythonOverridesAndPinOfflineServing() { + let base = ["HOME": "/Users/u", "PATH": "/x", "PYTHONPATH": "/evil", "PYTHONHOME": "/evil", + "VIRTUAL_ENV": "/venv", "CONDA_PREFIX": "/conda", "UV_INDEX_URL": "https://mirror", + "HF_HOME": "/hf"] + let server = MLXToolchainRules.serverEnvironment(base: base) + XCTAssertEqual(server["HF_HUB_OFFLINE"], "1") + XCTAssertEqual(server["HF_HOME"], "/hf", "a user-set cache location is honored") + XCTAssertNil(server["PYTHONPATH"]); XCTAssertNil(server["PYTHONHOME"]) + XCTAssertNil(server["VIRTUAL_ENV"]); XCTAssertNil(server["CONDA_PREFIX"]); XCTAssertNil(server["UV_INDEX_URL"]) + XCTAssertEqual(server["PATH"], "/usr/bin:/bin:/usr/sbin:/sbin") + + let download = MLXToolchainRules.downloadEnvironment(base: base) + XCTAssertNil(download["HF_HUB_OFFLINE"], "downloads must reach the network") + XCTAssertEqual(download["HF_HUB_DISABLE_TELEMETRY"], "1") + XCTAssertEqual(download["HF_HUB_DISABLE_PROGRESS_BARS"], "1") + + let install = MLXToolchainRules.installEnvironment(paths: paths, base: base) + XCTAssertEqual(install["UV_TOOL_DIR"], "/S/Cotabby/MLX/tools") + XCTAssertEqual(install["UV_TOOL_BIN_DIR"], "/S/Cotabby/MLX/bin") + XCTAssertEqual(install["UV_PYTHON_INSTALL_DIR"], "/S/Cotabby/MLX/python") + XCTAssertEqual(install["UV_CACHE_DIR"], "/S/Cotabby/MLX/cache") + XCTAssertEqual(install["UV_NO_CONFIG"], "1") + XCTAssertEqual(install["UV_PYTHON_PREFERENCE"], "only-managed") + XCTAssertEqual(install["UV_COMPILE_BYTECODE"], "1") + XCTAssertNil(install["UV_INDEX_URL"], "a user's uv mirror setting must not redirect Cotabby's install") + XCTAssertEqual(MLXToolchainRules.installArguments(), ["tool", "install", "mlx-lm==0.32.0", "--python", "3.12"]) + } + + func test_procArgsParsing() { + var bytes: [UInt8] = [] + withUnsafeBytes(of: Int32(3).littleEndian) { bytes += $0 } + bytes += Array("/usr/bin/python3".utf8) + [0, 0, 0, 0] + for arg in ["python3", "/S/MLX/bin/mlx_lm.server", "--port"] { bytes += Array(arg.utf8) + [0] } + bytes += Array("ENV=1".utf8) + [0] + XCTAssertEqual(MLXToolchainRules.parseProcArgs(bytes), ["python3", "/S/MLX/bin/mlx_lm.server", "--port"]) + XCTAssertNil(MLXToolchainRules.parseProcArgs([1, 0])) + } + + func test_isOurServerNeedsTheScriptAndLoopbackHost() { + XCTAssertTrue(MLXToolchainRules.isOurServer(arguments: + ["/p/python", "/S/MLX/bin/mlx_lm.server", "--model", "a/b", "--host", "127.0.0.1", "--port", "1"])) + XCTAssertFalse(MLXToolchainRules.isOurServer(arguments: + ["/p/python", "/S/MLX/bin/mlx_lm.server", "--host", "0.0.0.0"])) + XCTAssertFalse(MLXToolchainRules.isOurServer(arguments: ["/usr/bin/vim", "--host", "127.0.0.1"])) + } +} diff --git a/CotabbyTests/Support/Settings/SettingsAttentionEvaluatorTests.swift b/CotabbyTests/Support/Settings/SettingsAttentionEvaluatorTests.swift index acc37560d..bd64bd0d9 100644 --- a/CotabbyTests/Support/Settings/SettingsAttentionEvaluatorTests.swift +++ b/CotabbyTests/Support/Settings/SettingsAttentionEvaluatorTests.swift @@ -27,6 +27,21 @@ final class SettingsAttentionEvaluatorTests: XCTestCase { XCTAssertEqual(SettingsAttentionEvaluator.categoriesNeedingAttention(makeInputs()), []) } + func test_mlxProblemFlagsEngineAndModel() { + let base = SettingsAttentionEvaluator.Inputs( + permissionsGranted: true, + selectedEngine: .mlx, + foundationModelAvailable: true, + llamaRuntimeFailedReason: nil, + endpointConfigurationError: nil, + endpointConnectionFailedReason: nil + ) + XCTAssertEqual(SettingsAttentionEvaluator.categoriesNeedingAttention(base), []) + var broken = base + broken.mlxProblem = "MLX is not installed." + XCTAssertEqual(SettingsAttentionEvaluator.categoriesNeedingAttention(broken), [.engineAndModel]) + } + func test_missingPermissions_flagsPermissionsPane() { let categories = SettingsAttentionEvaluator.categoriesNeedingAttention( makeInputs(permissionsGranted: false) diff --git a/CotabbyTests/Support/Settings/SuggestionEngineSelectionPolicyTests.swift b/CotabbyTests/Support/Settings/SuggestionEngineSelectionPolicyTests.swift index 9e928fe7b..b559117ae 100644 --- a/CotabbyTests/Support/Settings/SuggestionEngineSelectionPolicyTests.swift +++ b/CotabbyTests/Support/Settings/SuggestionEngineSelectionPolicyTests.swift @@ -15,16 +15,33 @@ final class SuggestionEngineSelectionPolicyTests: XCTestCase { } func test_isSelectable_otherEnginesIgnoreAppleAvailability() { - for engine in [SuggestionEngineKind.llamaOpenSource, .openAICompatible] { + for engine in [SuggestionEngineKind.llamaOpenSource, .openAICompatible, .mlx] { for available in [true, false] { XCTAssertTrue( - SuggestionEngineSelectionPolicy.isSelectable(engine, foundationModelAvailable: available), + SuggestionEngineSelectionPolicy.isSelectable( + engine, foundationModelAvailable: available, isAppleSilicon: true + ), "\(engine) available=\(available)" ) } } } + func test_mlxNeedsAppleSilicon() { + XCTAssertTrue(SuggestionEngineSelectionPolicy.isSelectable( + .mlx, foundationModelAvailable: false, isAppleSilicon: true)) + XCTAssertFalse(SuggestionEngineSelectionPolicy.isSelectable( + .mlx, foundationModelAvailable: true, isAppleSilicon: false)) + XCTAssertEqual( + SuggestionEngineSelectionPolicy.pickerLabel( + for: .mlx, foundationModelAvailable: true, isAppleSilicon: false), + "MLX (Unavailable)" + ) + // Intel does not affect the engines that do not need MLX. + XCTAssertTrue(SuggestionEngineSelectionPolicy.isSelectable( + .llamaOpenSource, foundationModelAvailable: false, isAppleSilicon: false)) + } + func test_pickerLabel_marksOnlyUnselectableEngines() { XCTAssertEqual( SuggestionEngineSelectionPolicy.pickerLabel(for: .appleIntelligence, foundationModelAvailable: false), diff --git a/CotabbyTests/Support/Settings/SuggestionSettingsStoreTests.swift b/CotabbyTests/Support/Settings/SuggestionSettingsStoreTests.swift index 2bc6cdeb7..5d583aa03 100644 --- a/CotabbyTests/Support/Settings/SuggestionSettingsStoreTests.swift +++ b/CotabbyTests/Support/Settings/SuggestionSettingsStoreTests.swift @@ -728,6 +728,10 @@ final class SuggestionSettingsStoreTests: XCTestCase { store.saveOpenAICompatibleBaseURL("https://example.com/v1") store.saveOpenAICompatibleModelName("remote-model") store.saveOpenAICompatibleAPIMode(.completions) + store.saveMLXModelID("mlx-community/gemma-4-e4b-5bit") + store.saveMLXPromptCacheBytes(2 << 30) + store.saveBatteryMLXModelID("mlx-community/gemma-4-e2b-6bit") + store.savePluggedInMLXModelID("mlx-community/gemma-4-e4b-5bit") store.saveSelectedWordCountPreset(.fourToSeven) store.saveUsingCustomWordCountRange(true) store.saveCustomWordCountRange(low: 3, high: 9) diff --git a/CotabbyTests/Support/Suggestion/Request/DebouncePolicyTests.swift b/CotabbyTests/Support/Suggestion/Request/DebouncePolicyTests.swift index f184073dd..3c1d976db 100644 --- a/CotabbyTests/Support/Suggestion/Request/DebouncePolicyTests.swift +++ b/CotabbyTests/Support/Suggestion/Request/DebouncePolicyTests.swift @@ -60,4 +60,9 @@ final class DebouncePolicyTests: XCTestCase { // Without a latency, a configured fallback above the 180 ms floor wins. assertDebounce([(nil, 250)], fallback: 250, engine: .openAICompatible) } + + func test_mlxUsesTheLocalDebounce() { + XCTAssertEqual(DebouncePolicy.milliseconds(lastGenerationLatencyMilliseconds: 60, fallback: 40, engine: .mlx), 15) + XCTAssertEqual(DebouncePolicy.milliseconds(lastGenerationLatencyMilliseconds: nil, fallback: 40, engine: .mlx), 40) + } } diff --git a/CotabbyTests/Support/Suggestion/Request/SuggestionRequestFactoryTests.swift b/CotabbyTests/Support/Suggestion/Request/SuggestionRequestFactoryTests.swift index 6f07726fd..d92b2bee4 100644 --- a/CotabbyTests/Support/Suggestion/Request/SuggestionRequestFactoryTests.swift +++ b/CotabbyTests/Support/Suggestion/Request/SuggestionRequestFactoryTests.swift @@ -35,9 +35,25 @@ final class SuggestionRequestFactoryTests: XCTestCase { ) } + func test_buildRequest_mlxKeepsOnDeviceContext() { + let context = CotabbyTestFixtures.focusedInputContext( + precedingText: "We are meeting ", + trailingText: "LOCAL_DOCUMENT_TAIL" + ) + let result = SuggestionRequestFactory.buildRequest( + context: context, + settings: CotabbyTestFixtures.settingsSnapshot(selectedEngine: .mlx), + configuration: .standard, + historyExamples: ["HISTORY_EXAMPLE"] + ) + XCTAssertEqual(result.request.historyExamples, ["HISTORY_EXAMPLE"]) + XCTAssertTrue(result.request.prompt.contains("LOCAL_DOCUMENT_TAIL")) + XCTAssertTrue(result.request.prompt.hasSuffix("We are meeting ")) + } + func test_localScreenContextExceedsOldCapButEndpointKeepsLegacyPrompt() { let screen = String(repeating: "Project discussion and meeting agenda. ", count: 120) - for engine in [SuggestionEngineKind.llamaOpenSource, .appleIntelligence, .openAICompatible] { + for engine in [SuggestionEngineKind.llamaOpenSource, .appleIntelligence, .mlx, .openAICompatible] { let result = SuggestionRequestFactory.buildRequest( context: CotabbyTestFixtures.focusedInputContext(precedingText: "Please send "), settings: CotabbyTestFixtures.settingsSnapshot(selectedEngine: engine), diff --git a/README.md b/README.md index bb0040da5..28d2323b3 100644 --- a/README.md +++ b/README.md @@ -73,20 +73,24 @@ required. An optional OpenAI-compatible engine can connect to a server you confi Privacy is the whole point, so Cotabby's default engines keep generation on your Mac: -- Apple Intelligence and Open Source generation run on-device. +- Apple Intelligence and Open Source generation run on-device. The optional MLX engine also runs on + your Mac, in an `mlx_lm.server` process that listens only on 127.0.0.1. - The optional OpenAI-compatible engine sends a bounded request only to the endpoint you configure; that endpoint can be loopback, on your local network, or a public HTTPS service. - No analytics, no telemetry, no crash reporting. - A normal install never writes what you type to disk. -- Apart from a configured endpoint, the network is used for model downloads and update checks, not - suggestion generation. +- Apart from a configured endpoint, the network is used for model downloads, the optional MLX + install, and update checks, not suggestion generation. ## Engines -Cotabby generates suggestions in three ways. You choose which in Settings → Engine: +Cotabby generates suggestions in four ways. You choose which in Settings → Engine: - **Apple Intelligence** — Apple's model, built into macOS 26 or later on supported Macs. Nothing to download. - **Open Source** — a small AI model you download that runs entirely on your Mac. Works on any supported Mac (macOS 14+), with or without Apple Intelligence. +- **MLX** (Apple Silicon, optional) — an MLX-format model run by Apple's + [mlx-lm](https://github.com/ml-explore/mlx-lm). Cotabby installs a pinned mlx-lm into its own + folder only when you click Install, and uses models from the Hugging Face cache. - **OpenAI-compatible** — a completion or chat endpoint you configure, including local Ollama, another LAN host, or a public HTTPS service. Endpoint credentials are stored in Keychain. diff --git a/SOURCE_LAYOUT.md b/SOURCE_LAYOUT.md index 01dd59ace..62f7e9b4c 100644 --- a/SOURCE_LAYOUT.md +++ b/SOURCE_LAYOUT.md @@ -63,6 +63,7 @@ Cotabby/ │ ├── Runtime/ │ │ ├── AppleIntelligence/ Foundation Models availability and engine │ │ ├── Llama/ llama engine, manager, and native core +│ │ ├── MLX/ mlx-lm install, mlx_lm.server lifecycle, MLX model files │ │ └── OpenAICompatible/ endpoint transport, credentials, and engine │ ├── Spelling/ live spell-checking services │ ├── Suggestion/ @@ -92,6 +93,7 @@ Cotabby/ │ ├── Prompting/ Apple and base-model prompt rendering │ ├── Runtime/ │ │ ├── Hardware/ device capability and resource sampling +│ │ ├── MLX/ MLX cache scan, toolchain rules, server decision │ │ ├── BundledRuntimeLocator.swift │ │ ├── DecodeStopPolicy.swift │ │ ├── DownloadOutcomeClassifier.swift